DEV Community

Cover image for Learning Elixir: Binaries and Bitstrings
João Paulo Abreu
João Paulo Abreu

Posted on

Learning Elixir: Binaries and Bitstrings

Binaries and bitstrings can be thought of as digital building blocks where each block can hold exactly the amount of information you specify, down to the individual bit. I like to think of a bitstring as a precise digital ruler where you can measure and cut data to exact specifications - <<3::4>> allocates exactly 4 bits for the number 3, while <<42>> uses the standard 8-bit byte. Unlike higher-level data structures that abstract away the underlying representation, binaries give you direct control over how data is stored and manipulated at the byte level, making them useful for file formats and data processing. While this low-level control requires more precision and understanding, it also provides capabilities for data processing and pattern matching on binary streams. In this article, we'll explore how binaries and bitstrings work, their pattern matching capabilities, and techniques that are valuable in Elixir development.

Note: The examples in this article use Elixir 1.18.4. While most operations should work across different versions, some functionality might vary.

Table of Contents

Introduction

Binaries and bitstrings are Elixir's way of working with raw data at the bit and byte level. They're useful for string processing, file manipulation, and situations where you need precise control over data representation.

Some characteristics I've noticed about binaries and bitstrings:

  • Precise control: Specify exact bit sizes for optimal memory usage
  • Pattern matching power: Extract and validate data with elegant syntax
  • UTF-8 native: Excellent support for international text processing
  • Memory aware: Good memory representation and operations
  • Format friendly: Suitable for working with binary formats
  • Flexible: From single bits to data structures

Binaries and bitstrings are useful for data precision in cases like:

  • Network packets: <<version::4, type::4, length::16, data::binary>>
  • File headers: <<magic_number::32, version::8, flags::8>>
  • Image pixels: <<red::8, green::8, blue::8>>

Binaries are helpful when you need to:

  • Process binary data with specific byte layouts
  • Parse binary file formats
  • Handle international text with proper UTF-8 support
  • Create domain-specific binary formats
  • Transform and manipulate data at the byte level

Let's explore how they work!

Understanding Binaries and Bitstrings

Fundamental Concepts

The relationship between bitstrings and binaries can be understood this way:

# A bitstring is a contiguous sequence of bits in memory
bitstring_4_bits = <<3::4>>    # Exactly 4 bits
bitstring_12_bits = <<3::4, 7::8>>  # 12 bits total (4 + 8)

# A binary is a bitstring where the number of bits is divisible by 8
binary_8_bits = <<42>>         # 8 bits (1 byte)
binary_16_bits = <<42, 24>>    # 16 bits (2 bytes)

# Every binary is a bitstring, but not every bitstring is a binary
is_bitstring(<<3::4>>)         # true
is_binary(<<3::4>>)           # false - not divisible by 8

is_bitstring(<<42>>)          # true  
is_binary(<<42>>)             # true - 8 bits, divisible by 8
Enter fullscreen mode Exit fullscreen mode

Understanding Bitstring Syntax

Learning to "read" bitstring syntax helps understand how data is stored and manipulated at the bit level.

Basic Syntax Patterns

# The pattern: <<value::size>>
# "Store [value] using exactly [size] bits"

<<3::4>>        # "Store 3 using 4 bits"
<<42::8>>       # "Store 42 using 8 bits" (default, same as <<42>>)
<<1000::16>>    # "Store 1000 using 16 bits"
<<7::size(12)>> # "Store 7 using 12 bits" (variable size)
Enter fullscreen mode Exit fullscreen mode

Bit Patterns and Storage

# 3::4 means "store the number 3 using exactly 4 bits"
# 3 in binary: 0011 (4 bits)
<<3::4>>  # Stores: 0011

# 7::8 means "store the number 7 using exactly 8 bits"
# 7 in binary: 00000111 (8 bits)
<<7::8>>  # Stores: 00000111

# 15::4 means "store 15 using 4 bits"
# 15 in binary: 1111 (4 bits) - fits perfectly
<<15::4>>  # Stores: 1111

# Visualizing the actual bit patterns (for complete bytes)
<<byte>> = <<15>>
Integer.to_string(byte, 2) |> String.pad_leading(8, "0")
# Result: "00001111" - shows the actual bit representation
Enter fullscreen mode Exit fullscreen mode

Handling Overflow and Truncation

# What happens when a value doesn't fit?
# 16::4 means "store 16 using 4 bits"
# 16 in binary: 10000 (5 bits), but we only have 4 bits!
# Gets truncated to: 0000 (keeps only the lowest 4 bits)
<<16::4>>  # Stores: 0000 (truncation occurs)

# This truncation behavior is predictable and follows bitwise AND with max value:
# 17::4 -> 0001 (17 & 15 = 1, where 15 is max 4-bit value)
# 18::4 -> 0010 (18 & 15 = 2)
Enter fullscreen mode Exit fullscreen mode

Complex Concatenations

# Creating a complex bitstring by concatenating different sized parts
complex_bits = <<3::4, 7::8>>  # 4 bits + 8 bits = 12 bits total

# Understanding what happens step by step:
# 3 in 4 bits:  0011
# 7 in 8 bits:  00000111
# Concatenated: 0011 00000111 (12 bits total)

# Since 12 bits isn't divisible by 8, Elixir groups them:
# First 8 bits:  00110000 → decimal 48
# Last 4 bits:   0111 → decimal 7 (partial byte)
# Result: <<48, 7::size(4)>>

bit_size(complex_bits)   # 12 bits total
byte_size(complex_bits)  # 2 bytes (rounded up)

# Verify the individual components:
part1 = Integer.to_string(3, 2) |> String.pad_leading(4, "0")    # "0011"
part2 = Integer.to_string(7, 2) |> String.pad_leading(8, "0")    # "00000111"

# The concatenation "001100000111" gets split as:
# "00110000" (8 bits) = 48 decimal
# "0111" (4 bits) = 7 decimal
# Hence: <<48, 7::size(4)>>
Enter fullscreen mode Exit fullscreen mode

Reading IEx Output

IEx bitstring results follow predictable patterns:

<<42>>            # Simple binary: all bits divisible by 8
<<3::size(4)>>    # Bitstring: 4 bits, not divisible by 8
<<48, 7::size(4)>> # Mixed: full byte + partial bits

# The pattern: <<full_bytes..., partial_value::size(remaining_bits)>>
Enter fullscreen mode Exit fullscreen mode

This bit-level precision makes bitstrings useful for precise data control.

Key insight: When bits don't align to byte boundaries (8-bit multiples), Elixir shows the result as complete bytes plus remaining bits. So <<3::4, 7::8>> creates 12 bits total, displayed as one complete byte (48) plus 4 remaining bits (7).

Testing in IEx:

iex> bitstring_4_bits = <<3::4>>
<<3::size(4)>>

iex> bitstring_12_bits = <<3::4, 7::8>>  # 4 + 8 = 12 bits total
<<48, 7::size(4)>>

iex> binary_8_bits = <<42>>
"*"

iex> binary_16_bits = <<42, 24>>
<<42, 24>>

iex> is_bitstring(<<3::4>>)
true

iex> is_binary(<<3::4>>)
false

iex> is_bitstring(<<42>>)
true

iex> is_binary(<<42>>)
true

iex> <<byte>> = <<15>>
<<15>>

iex> Integer.to_string(byte, 2) |> String.pad_leading(8, "0")
"00001111"

iex> complex_bits = <<3::4, 7::8>>
<<48, 7::size(4)>>

iex> bit_size(complex_bits)
12

iex> byte_size(complex_bits)
2

iex> part1 = Integer.to_string(3, 2) |> String.pad_leading(4, "0")
"0011"

iex> part2 = Integer.to_string(7, 2) |> String.pad_leading(8, "0")
"00000111"

iex> <<42::16>>
<<0, 42>>

iex> is_binary(<<42::16>>)
true
Enter fullscreen mode Exit fullscreen mode

Size Functions and Inspection

data = <<1, 2, 3, 255>>

# Get size information
bit_size(data)         # 32 (4 bytes × 8 bits/byte)
byte_size(data)        # 4 bytes

# For non-byte-aligned bitstrings
partial = <<1::3, 0::2, 1::3>>  # 8 bits total but constructed from parts
bit_size(partial)      # 8
byte_size(partial)     # 1 (rounds up to nearest byte)

# Size functions work in guards
defmodule BinaryValidator do
  def small_binary?(data) when byte_size(data) <= 10, do: true
  def small_binary?(_), do: false

  def exact_size?(data) when bit_size(data) == 64, do: true
  def exact_size?(_), do: false
end
Enter fullscreen mode Exit fullscreen mode

Testing in IEx:

iex> data = <<1, 2, 3, 255>>
<<1, 2, 3, 255>>

iex> bit_size(data)
32

iex> byte_size(data)
4

iex> partial = <<1::3, 0::2, 1::3>>
"!"

iex> bit_size(partial)
8

iex> byte_size(partial)
1
Enter fullscreen mode Exit fullscreen mode

Memory Representation and Efficiency

# Binaries are stored in memory
large_binary = :crypto.strong_rand_bytes(1024)  # 1KB of random data
byte_size(large_binary)  # 1024

# Binary concatenation creates a new binary
combined = <<1, 2>> <> <<3, 4>>  # <<1, 2, 3, 4>>

# Pattern matching can be used to avoid copying
<<prefix::binary-size(2), suffix::binary>> = large_binary
byte_size(prefix)   # 2
byte_size(suffix)   # 1022 (shares memory with original)

# Check if data is a proper binary vs bitstring
defmodule TypeChecker do
  def analyze(data) do
    %{
      is_bitstring: is_bitstring(data),
      is_binary: is_binary(data),
      bit_size: bit_size(data),
      byte_size: byte_size(data),
      is_proper_binary: is_binary(data)
    }
  end
end
Enter fullscreen mode Exit fullscreen mode

Testing in IEx:

iex> combined = <<1, 2>> <> <<3, 4>>
<<1, 2, 3, 4>>

iex> <<prefix::binary-size(2), suffix::binary>> = <<1, 2, 3, 4, 5>>
<<1, 2, 3, 4, 5>>

iex> prefix
<<1, 2>>

iex> suffix
<<3, 4, 5>>
Enter fullscreen mode Exit fullscreen mode

Binary Construction and Syntax

Basic Construction Patterns

# Default 8-bit integers
basic = <<1, 2, 3>>           # Three bytes
explicit = <<1::8, 2::8, 3::8>>  # Equivalent explicit form

# Different bit sizes
small = <<15::4>>             # 4 bits (fits perfectly)
large = <<1000::16>>          # 16 bits (2 bytes)

# Mixed sizes in one binary
mixed = <<1::4, 2::8, 3::4>>  # 16 bits total = 2 bytes

# Verify equivalence
basic == explicit             # true
bit_size(mixed)              # 16
byte_size(mixed)             # 2
Enter fullscreen mode Exit fullscreen mode

Testing in IEx:

iex> <<1, 2, 3>>
<<1, 2, 3>>

iex> <<1::8, 2::8, 3::8>>
<<1, 2, 3>>

iex> <<15::4>>
<<15::size(4)>>

iex> <<1000::16>>
<<3, 232>>

iex> mixed = <<1::4, 2::8, 3::4>>
<<16, 35>>

iex> bit_size(mixed)
16
Enter fullscreen mode Exit fullscreen mode

Advanced Size Specifications

# Variable sizes using variables
size = 12
variable_size = <<42::size(size)>>   # 12-bit value

# Size expressions (evaluated at compile time when possible)
header_size = 8
data_size = 24
packet = <<1::size(header_size), 12345::size(data_size)>>

# Working with different data types
float_data = <<3.14159::32-float>>   # 32-bit float
double_data = <<3.14159::64-float>>  # 64-bit float

# Signedness specifications (affects how negative numbers are stored)
signed_positive = <<100::8-signed>>    # Positive: stored as 100 (shows as "d")
signed_negative = <<-50::8-signed>>    # Negative: stored as 206 (two's complement of -50)
unsigned_value = <<206::8>>            # Unsigned: stored as 206 (same bytes, different meaning)

# Endianness control
big_endian = <<0x1234::16-big>>       # <<18, 52>>
little_endian = <<0x1234::16-little>> # <<52, 18>>
Enter fullscreen mode Exit fullscreen mode

Testing in IEx:

iex> size = 12
12

iex> <<42::size(size)>>
<<2, 10::size(4)>>

iex> <<3.14159::32-float>>
<<64, 73, 15, 208>>

iex> <<100::8-signed>>
"d"

iex> <<-50::8-signed>>
<<206>>

iex> <<206::8>>
<<206>>

iex> <<0x1234::16-big>>
<<18, 52>>

iex> <<0x1234::16-little>>
<<52, 18>>
Enter fullscreen mode Exit fullscreen mode

Understanding Signed vs Unsigned: IEx displays <<100::8-signed>> as "d" because byte 100 corresponds to the ASCII character 'd'. The signedness modifier affects how negative numbers are encoded (using two's complement) but doesn't change positive number storage. Both <<100::8-signed>> and <<100::8>> store identical byte values for positive numbers.

String and Character Handling

# Strings are UTF-8 encoded binaries
string = "Hello, 世界!"
is_binary(string)      # true
byte_size(string)      # 14 bytes (not 10 characters!)

# Manual UTF-8 construction
utf8_manual = <<72, 101, 108, 108, 111>>  # "Hello" in UTF-8
utf8_manual == "Hello"  # true

# Unicode code points
unicode_point = <<0x4E16::utf8, 0x754C::utf8>>  # 世界
unicode_point == "世界"  # true

# Null-terminated strings (C-style)
c_string = <<"Hello", 0>>    # Hello with null terminator
printable_part = binary_part(c_string, 0, byte_size(c_string) - 1)
Enter fullscreen mode Exit fullscreen mode

Testing in IEx:

iex> string = "Hello, 世界!"
"Hello, 世界!"

iex> byte_size(string)
14

iex> <<72, 101, 108, 108, 111>>
"Hello"

iex> <<0x4E16::utf8, 0x754C::utf8>>
"世界"

iex> c_string = <<"Hello", 0>>
<<72, 101, 108, 108, 111, 0>>

iex> binary_part(c_string, 0, byte_size(c_string) - 1)
"Hello"
Enter fullscreen mode Exit fullscreen mode

Building Complex Structures

defmodule DataBuilder do
  # Binary structure with header and payload
  def build_data(type, payload) when is_binary(payload) do
    version = 1
    length = byte_size(payload)
    checksum = calculate_checksum(payload)

    <<version::4, type::4, length::16, checksum::32, payload::binary>>
  end

  # Message encoding
  def encode_message(id, data) when is_list(data) do
    encoded_data = Enum.map(data, &encode_field/1)
    data_size = Enum.sum_by(encoded_data, &byte_size/1)

    [<<id::16, data_size::32>> | encoded_data]
    |> IO.iodata_to_binary()
  end

  # Binary tree node representation
  def encode_tree_node(value, left_present?, right_present?) do
    flags = 
      case {left_present?, right_present?} do
        {true, true} -> 3    # 11 in binary
        {true, false} -> 2   # 10 in binary  
        {false, true} -> 1   # 01 in binary
        {false, false} -> 0  # 00 in binary
      end

    <<flags::2, 0::6, value::32>>  # Flags + padding + value
  end

  defp calculate_checksum(data) do
    # Simple XOR checksum
    import Bitwise
    data
    |> :binary.bin_to_list()
    |> Enum.reduce(0, fn byte, acc -> bxor(byte, acc) end)
  end

  defp encode_field({:string, str}) do
    size = byte_size(str)
    <<1, size::16, str::binary>>
  end

  defp encode_field({:integer, int}) do
    <<2, int::32>>
  end

  defp encode_field({:float, f}) do
    <<3, f::32-float>>
  end
end
Enter fullscreen mode Exit fullscreen mode

Testing in IEx:

iex> data = DataBuilder.build_data(5, "Hello")
<<21, 0, 5, 0, 0, 0, 66, 72, 101, 108, 108, 111>>

iex> message = DataBuilder.encode_message(100, [{:string, "test"}, {:integer, 42}])
<<0, 100, 0, 0, 0, 11, 1, 0, 4, 116, 101, 115, 116, 2, 0, 0, 0, 42>>

iex> node = DataBuilder.encode_tree_node(1000, true, false)
<<128, 0, 0, 3, 232>>
Enter fullscreen mode Exit fullscreen mode

Pattern Matching with Binaries

Basic Pattern Matching

# Fixed-size pattern matching
<<a, b, c>> = <<1, 2, 3>>   # a=1, b=2, c=3

# Mixed with rest
<<first, rest::binary>> = <<1, 2, 3, 4>>
# first = 1, rest = <<2, 3, 4>>

# Specific sizes
<<header::binary-size(2), payload::binary>> = <<0xFF, 0xFE, 1, 2, 3>>
# header = <<255, 254>>, payload = <<1, 2, 3>>

# Multiple extractions
<<version::4, type::4, length::16, data::binary>> = <<0x12, 0x00, 0x05, 1, 2, 3, 4, 5>>
# version = 1, type = 2, length = 5, data = <<1, 2, 3, 4, 5>>
Enter fullscreen mode Exit fullscreen mode

Common Patterns

defmodule BinaryPatterns do
  # Variable-length data with prefixed size
  def parse_sized_string(<<size::16, string::binary-size(size), rest::binary>>) do
    {:ok, string, rest}
  end

  # Pattern matching with guards
  def validate_packet(<<version, type, _rest::binary>>)
      when version in 1..3 and type in 1..10 do
    {:ok, :valid_packet}
  end

  # Size-based validation
  def process_message(data) when byte_size(data) <= 64 do
    {:ok, "Processing small message"}
  end

  def process_message(_data) do
    {:error, "Message too large"}
  end
end
Enter fullscreen mode Exit fullscreen mode

Testing in IEx:

iex> <<a, b, c>> = <<1, 2, 3>>
<<1, 2, 3>>

iex> {a, b, c}
{1, 2, 3}

iex> <<header::binary-size(2), payload::binary>> = <<255, 254, 1, 2, 3>>
<<255, 254, 1, 2, 3>>

iex> {header, payload}
{<<255, 254>>, <<1, 2, 3>>}

iex> sized_data = <<5::16, "hello", "rest">>
<<0, 5, 104, 101, 108, 108, 111, 114, 101, 115, 116>>

iex> BinaryPatterns.parse_sized_string(sized_data)
{:ok, "hello", "rest"}

iex> packet = <<2, 5, "more data">>
<<2, 5, 109, 111, 114, 101, 32, 100, 97, 116, 97>>

iex> BinaryPatterns.validate_packet(packet)
{:ok, :valid_packet}

iex> BinaryPatterns.process_message("small")
{:ok, "Processing small message"}
Enter fullscreen mode Exit fullscreen mode

String Operations and UTF8 Handling

UTF-8 Pattern Matching

Working with international text requires special consideration for multi-byte characters:

# Incorrect way - treats bytes, not characters
<<first, rest::binary>> = "über"
first ==   # false! (first is 195, first byte of ü in UTF-8)

# Correct way - use utf8 modifier
<<first::utf8, rest::binary>> = "über"
first ==   # true! (first is the Unicode code point)

defmodule UTF8Handler do
  # Extract first character safely
  def first_char(<<char::utf8, _rest::binary>>), do: {:ok, char}
  def first_char(""), do: {:error, :empty_string}
  def first_char(_), do: {:error, :invalid_utf8}

  # Extract last character (more complex due to variable length)
  def last_char(string) when is_binary(string) do
    case String.reverse(string) do
      <<char::utf8, _rest::binary>> -> {:ok, char}
      "" -> {:error, :empty_string}
      _ -> {:error, :invalid_utf8}
    end
  end

  # Count actual characters vs bytes
  def analyze_string(string) do
    %{
      byte_size: byte_size(string),
      char_count: String.length(string),
      first_char: first_char(string),
      is_ascii: is_ascii?(string)
    }
  end

  # Check if string contains only ASCII characters
  defp is_ascii?(string) do
    string
    |> String.to_charlist()
    |> Enum.all?(fn char -> char >= 0 and char <= 127 end)
  end

  # Safe character iteration
  def extract_chars(binary, acc \\ [])

  def extract_chars(<<char::utf8, rest::binary>>, acc) do
    extract_chars(rest, [char | acc])
  end

  def extract_chars("", acc) do
    {:ok, Enum.reverse(acc)}
  end

  def extract_chars(_invalid, _acc) do
    {:error, :invalid_utf8}
  end
end
Enter fullscreen mode Exit fullscreen mode

Testing in IEx:

iex> <<first::utf8, rest::binary>> = "über"
"über"

iex> first == 
true

iex> rest
"ber"

iex> UTF8Handler.analyze_string("Hello")
%{byte_size: 5, char_count: 5, first_char: {:ok, 72}, is_ascii: true}

iex> UTF8Handler.analyze_string("世界")
%{byte_size: 6, char_count: 2, first_char: {:ok, 19990}, is_ascii: false}

iex> UTF8Handler.extract_chars("Héllo")
{:ok, [72, 233, 108, 108, 111]}
Enter fullscreen mode Exit fullscreen mode

String Construction and Manipulation

defmodule StringBuilder do
  # Build strings from Unicode code points
  def from_codepoints(codepoints) when is_list(codepoints) do
    try do
      binary = for cp <- codepoints, into: <<>>, do: <<cp::utf8>>
      {:ok, binary}
    rescue
      _ -> {:error, :invalid_codepoint}
    end
  end

  # Remove specific characters using binary patterns
  def remove_char(string, char_to_remove) do
    for <<char::utf8 <- string>>, char != char_to_remove, into: "", do: <<char::utf8>>
  end

  # Convert binary to hexadecimal representation
  def to_hex_string(binary) do
    for <<byte <- binary>>, into: "" do
      byte |> Integer.to_string(16) |> String.downcase() |> String.pad_leading(2, "0")
    end
  end
end
Enter fullscreen mode Exit fullscreen mode

Testing in IEx:

iex> StringBuilder.from_codepoints([72, 101, 108, 108, 111])
{:ok, "Hello"}

iex> StringBuilder.remove_char("Hello World", ?l)
"Heo Word"

iex> StringBuilder.to_hex_string(<<255, 0, 128>>)
"ff0080"
Enter fullscreen mode Exit fullscreen mode

Bitwise Operations and Manipulation

Basic Bitwise Operations

import Bitwise

# Basic operations
a = 0b1010  # 10 in decimal
b = 0b1100  # 12 in decimal

# Demonstrating bitwise operations
and_result = a &&& b        # 0b1000 = 8
or_result = a ||| b         # 0b1110 = 14
xor_result = bxor(a, b)     # 0b0110 = 6
not_result = bnot(a) &&& 0xFF # Limit to 8 bits = 245
left_shift = a <<< 2        # 0b101000 = 40
right_shift = a >>> 1       # 0b101 = 5
Enter fullscreen mode Exit fullscreen mode

Binary Data Operations

defmodule BitwiseOps do
  import Bitwise

  # Essential bit manipulation functions
  def set_bit(value, position), do: value ||| (1 <<< position)
  def clear_bit(value, position), do: value &&& bnot(1 <<< position)
  def bit_set?(value, position), do: (value &&& (1 <<< position)) != 0

  # XOR two binaries
  def xor_binaries(<<>>, <<>>), do: <<>>
  def xor_binaries(<<a, rest_a::binary>>, <<b, rest_b::binary>>) do
    <<bxor(a, b), xor_binaries(rest_a, rest_b)::binary>>
  end

  # Simple checksum
  def calculate_checksum(data) do
    for <<byte <- data>>, reduce: 0 do
      acc -> (acc + byte) &&& 0xFFFF
    end
  end
end
Enter fullscreen mode Exit fullscreen mode

Testing in IEx:

iex> import Bitwise
Bitwise

iex> a = 0b1010
10

iex> a &&& 0b1100
8

iex> a ||| 0b1100
14

iex> BitwiseOps.calculate_checksum(<<1, 2, 3, 4, 5>>)
15

iex> BitwiseOps.set_bit(0b1010, 0)
11

iex> BitwiseOps.bit_set?(0b1010, 1)
true
Enter fullscreen mode Exit fullscreen mode

Binary Comprehensions and Generators

Basic Binary Comprehensions

# Transform bytes in a binary
data = <<1, 2, 3, 4, 5>>
doubled = for <<byte <- data>>, into: <<>>, do: <<byte * 2>>
# Result: <<2, 4, 6, 8, 10>>

# Filter while processing
filtered = for <<byte <- data>>, byte > 2, into: <<>>, do: <<byte>>
# Result: <<3, 4, 5>>
Enter fullscreen mode Exit fullscreen mode

Common Processing Patterns

defmodule BinaryProcessor do
  # Remove specific bytes
  def remove_bytes(data, bytes_to_remove) do
    for <<byte <- data>>, byte not in bytes_to_remove, into: <<>>, do: <<byte>>
  end

  # Convert to hexadecimal
  def to_hex(data) do
    for <<byte <- data>>, into: "" do
      byte |> Integer.to_string(16) |> String.downcase() |> String.pad_leading(2, "0")
    end
  end
end
Enter fullscreen mode Exit fullscreen mode

Testing in IEx:

iex> data = <<1, 2, 3, 4, 5>>
<<1, 2, 3, 4, 5>>

iex> for <<byte <- data>>, into: <<>>, do: <<byte * 2>>
<<2, 4, 6, 8, 10>>

iex> for <<byte <- data>>, byte > 2, into: <<>>, do: <<byte>>
<<3, 4, 5>>

iex> BinaryProcessor.remove_bytes(data, [2, 4])
<<1, 3, 5>>

iex> BinaryProcessor.to_hex(<<255, 0, 128>>)
"ff0080"
Enter fullscreen mode Exit fullscreen mode

Conclusion

Binaries and bitstrings are a fascinating feature of Elixir for working with data at the bit and byte level. In this article, I explored:

  • How bitstrings and binaries work internally with precise bit-level control
  • Binary construction syntax for creating data structures
  • Pattern matching techniques for elegant data extraction and validation
  • String operations with proper UTF-8 handling for international text
  • Bitwise operations for low-level data manipulation
  • Binary comprehensions for data transformations

Some things I learned:

  • Precise control: Bitstrings allow exact specification of data representation down to individual bits
  • Pattern matching power: Binary pattern matching provides elegant solutions for data parsing and validation
  • UTF-8 support: Excellent support for international text with proper character handling
  • Memory efficiency: Smart memory sharing and efficient operations
  • Comprehension support: Powerful transformation capabilities with binary comprehensions

Binaries are one way Elixir handles data processing that I've found particularly interesting. While higher-level data structures like lists and maps provide developer ergonomics, binaries can provide the precision needed for file format processing and data manipulation. Their integration with pattern matching can make parsing tasks more readable and maintainable.

Learning about binaries has changed how I approach data processing in Elixir. They've proven to be a useful feature for building systems that handle specialized data formats and binary processing tasks.

I've found the combination of safety and expressiveness makes Elixir's binary handling a valuable tool for application development that needs precise data control.

Further Reading

Next Steps

With a solid understanding of binaries and bitstrings, the next topic is Ranges in Elixir. Ranges provide an elegant way to represent sequences of numbers, enabling iteration patterns, enumeration operations, and slice operations that complement the precise data control offered by binaries.

The next article will explore:

  • Creating and working with range syntax and operations
  • Range enumeration patterns and lazy evaluation benefits
  • Using ranges for slicing collections and pattern matching
  • Performance characteristics and memory efficiency of ranges
  • Applications in data processing and algorithms

Ranges are another useful building block in Elixir, providing ways to work with sequences that scale from small iterations to large data processing tasks!

Top comments (0)