DEV Community: Anton Maminov

Announcing Lustra: a Postgres ORM for Crystal

Anton Maminov — Mon, 06 Oct 2025 16:39:52 +0000

TL;DR: Lustra started as a fork of Clear at version 0.8, and it is not compatible with later Clear releases. Over time it evolved into an independent project. To keep it compatible with newer Crystal versions, I continued development, added missing features, improved existing ones, and expanded test coverage. Today Lustra stands as its own project.

When I started building shards.info, I had to make an important choice: which ORM to use with Crystal. Coming from a Ruby on Rails background, I was naturally looking for something similar to ActiveRecord — straightforward, feature-rich, and reliable. I reviewed several options available at the time: Jennifer, Crecto, Granite, and Clear. After evaluating them, I chose Clear. At the time, it was the most advanced ORM in the Crystal ecosystem. Even though it has been limited to Postgres support, I didn’t plan to use other databases, so this limitation was never a drawback for me.

Clear later introduced a new major version with substantial API and behavior changes. Some of those changes (and a few bugs at the time) broke previously working code in my project. My system was already running smoothly, and I wasn’t ready to trade stability for churn.

To keep my project alive - I forked it. That fork is now Lustra.

What is Lustra?

Lustra is an ORM built specifically for PostgreSQL in Crystal, maintained independently.
It aims to provide a stable, reliable, and Crystal-compatible ORM for Postgres database.

Why Lustra?

Proven capabilities: Associations, query builder, type safety - the same qualities that originally attracted me to Clear.
Stable API: I don’t plan to change the API in the future, so existing projects can rely on consistent behavior.
Independent evolution: I’ve ensured compatibility with new Crystal versions, added new features, and expanded test coverage.

The name

Why “Lustra”?
Because the word means “chandelier” in several languages. To me, a chandelier looks like a database schema: a structured network of connections. It also carries the idea of clarity and shine.

Personal reflection

Working on Lustra started as a practical need to keep my project running. Over time, it became an opportunity to explore Crystal and ORM internals more deeply. Lustra represents my preference for tools that are stable and predictable. Maintaining it has also shown me how consistent effort can turn a fork into an independent project.

What’s next?

Lustra will continue as a stable Postres ORM for Crystal developers.

If you’re looking for a dependable ORM for your Crystal projects, give Lustra a try.

👉 https://github.com/crystal-garage/lustra

Semaphores in Crystal

Anton Maminov — Sun, 23 Mar 2025 16:45:26 +0000

TL;DR

Need to limit how many tasks run at once in your Crystal app? You can implement semaphore-like control using WaitGroup. This article shows how to manage concurrent downloads with just a few lines of code.

Why I wrote this article

When I started working on a problem that required limiting concurrent downloads in Crystal, I didn't know what semaphores were at all. I just needed a way to control how many threads run at the same time. While searching for a solution, I didn't find any information about using semaphores in Crystal—so I decided to write this article and share what I've learned.

What is a semaphore?

"Think of semaphores as bouncers at a nightclub. There are a dedicated number of people that are allowed in the club at once. If the club is full no one is allowed to enter, but as soon as one person leaves another person might enter."

— Stack Overflow comment https://stackoverflow.com/questions/34519/what-is-a-semaphore/40473#40473

In simpler terms, a semaphore controls access to resources by setting a limit on how many tasks can run concurrently.

Implementing semaphore-like behavior in Crystal

Crystal doesn't have built-in semaphores, but we can achieve similar control using Atomic counters and WaitGroup.

WaitGroup is a simpler and more efficient alternative to using a Channel(Nil).

Here's a practical example of concurrent file downloads:

require "http/client"
require "wait_group"

# Only used to synchronize console output, not part of concurrency control logic
mutex = Mutex.new

# Number of concurrent downloads allowed at a time
threads = 4

# List of URLs to be downloaded with random sizes
download_urls = [] of {String, Int32}

15.times do |i|
  size = Random.rand(1_000_000..20_000_000) # Random size between 1MB and 20MB
  url = "http://speedtest.astra.in.ua.prod.hosts.ooklaserver.net:8080/download?size=#{size}"
  download_urls << {url, size}
end

# Atomic counter to track the number of currently active downloads
active_downloads = Atomic(Int32).new(0)

puts "⬇️ Starting downloads with WaitGroup..."

WaitGroup.wait do |wg|
  # Process each download URL with concurrency control
  download_urls.each_with_index do |(url, size), index|
    # Block execution if the number of active downloads reaches the limit
    while active_downloads.get >= threads
      sleep 10.milliseconds
    end

    active_downloads.add(1) # Increase the count of active downloads

    start_time = Time.local

    mutex.synchronize do
      puts "[#{start_time}] 🟢 Starting download #{index + 1} (Size: #{size.humanize_bytes}) | Active downloads: #{active_downloads.get}"
    end

    wg.spawn do
      HTTP::Client.get(url)
    rescue ex
      mutex.synchronize do
        puts "❌ Download failed: #{url} - #{ex.message}"
      end
    ensure
      active_downloads.sub(1) # Decrease the count of active downloads

      end_time = Time.local
      duration = end_time - start_time

      mutex.synchronize do
        puts "[#{end_time}] 🔴 Finished download #{index + 1} (Size: #{size.humanize_bytes}) | Active downloads: #{active_downloads.get} | Duration: #{duration.total_seconds.round(2)}s"
      end
    end
  end
end

puts "✅ All downloads completed!"

How this code works

Atomic counter tracks how many downloads are currently active.
WaitGroup ensures the program waits for all downloads to finish.
The loop pauses briefly (sleep 10.milliseconds) when the maximum allowed concurrent downloads (threads) is reached.

Note: The Mutex used in the example is not part of the concurrency control mechanism. It's only used to prevent overlapping output in the terminal when multiple fibers print simultaneously. You can safely remove it if you don't care about how the logs look. Alternatively, for a more structured approach to logging, you can use Crystal's built-in Log module to print status information.

Example output

Below is a screenshot of the actual output showing how downloads start and complete asynchronously:

As you can see, download 1 (started first) finishes later than the last download 15 starts. This is because downloads run concurrently, not sequentially. Some files are smaller or download faster, allowing other downloads to begin as soon as a slot opens. This demonstrates the efficient use of concurrency, where tasks don't block one another unnecessarily.

Happy Crystalling!

Crystal 1.4 Enumerable#tally now accepts an optional hash to count occurrences

Anton Maminov — Mon, 11 Apr 2022 17:11:51 +0000

Enumeration refers to traversing, searching, filtering, and querying objects.

In Crystal, we have classes like Array, Hash, Range, etc. which get their enumeration features by including the Enumerable module.

This Enumerable module provides various methods e.g., #select, #map, #reduce, and #uniq which we frequently use, and less known #tally about which this post is.

#tally counts the number of occurrences of an element in a collection and returns a hash containing the count for all elements.

['a', 'b', 'c', 'b'].tally # => {'a'=>1, 'b'=>2, 'c'=>1}

Example

Consider the example of a list of words, and you want to count the total number of each char.

This type of problem is a perfect candidate to leverage our #tally method to calculate the total quantity of chars.

Consider you have words as follows:

word_one = "crystal"
word_two = "ruby"

Before

Calculating the quantity of chars in each word:

tally_one = word_one.chars.tally
# => {'c' => 1, 'r' => 1, 'y' => 1, 's' => 1, 't' => 1, 'a' => 1, 'l' => 1}

tally_two = word_two.chars.tally
# => {'r' => 1, 'u' => 1, 'b' => 1, 'y' => 1}

total_tally = tally_one.merge(tally_two) { |k, v1, v2| v1 + v2 }
# => {'c' => 1, 'r' => 2, 'y' => 2, 's' => 1, 't' => 1, 'a' => 1, 'l' => 1, 'u' => 1, 'b' => 1}

We have merged tally_one and tally_two and summed hash values to get the combined count of the chars.

After

To calculate the total, Crystal 1.4 Enumerable#tally now accepts an optional hash to count occurrences.

In this case, we can store the running tally of the number of items and pass it to the #tally method.

word_one = "crystal"
word_two = "ruby"

total_tally = {} of Char => Int32

word_one.chars.tally(total_tally)
word_two.chars.tally(total_tally)

total_tally
=> {'c' => 1, 'r' => 2, 'y' => 2, 's' => 1, 't' => 1, 'a' => 1, 'l' => 1, 'u' => 1, 'b' => 1}

As you can see in the above example in Crystal 1.4, we have passed the total_tally hash as an argument to the #tally method that stores the count of chars.

Finally, we can simplify the code to one line:

words = ["crystal", "ruby"]

total_tally = words.reduce({} of Char => Int32) { |acc, word| word.chars.tally(acc) }

Crest 1.0.0 Release

Anton Maminov — Thu, 14 Oct 2021 11:29:46 +0000

This article is about Crest — HTTP and REST client for Crystal programming language.

A little about me

My name is Anton and I'm a self-taught full-stack Ruby on Rails developer.

Back in 2017, I had a desire to learn a new programming language.

Crystal caught my attention because of it's similarity to Ruby and because "Fast as C, Slick as Ruby ©".

Literally you can run the same code in Ruby and in Crystal, and it will work.

Unlike Ruby, Crystal is a typed language, but most of the time it's not required to specify types.

Crystal comes with a great standard library and tools. It has almost all the stuff you need to build applications.
CSV, XML, YAML, JSON, HTTP server/client, and even WebSocket are bundled with Crystal itself, making it super simple to start building something.

Project history

For practical reasons, I decided to re-implement Ruby's rest-client gem to dive into the Crystal.

I started the Crest project at April 2017 when Crystal was 0.21.1.

And in a 7 months I has working implementation with next features:

redirects support
proxy support
support nested hashes for headers, cookies, query params, and form data.
multipart file uploads
logging

The first stable version 0.9.2 was released on 1 Nov 2017.

After more than 4 years of development and 48 releases, I'm happy to announce version Crest 1.0.0.

Example

The common case of HTTP clients is sending a form data to a server.

Let's implement curl command below:

curl -X POST https://httpbin.org/post\?author\=John+Doe\&offset\=20 -F 'user[name]=Tom' -F 'user[file]=@/path/to/file.png' -H 'Foo: bar' --cookie 'foo=bar' -A 'Mozilla/5.0'

Here is an example showing how to do this in pure Crystal:

require "http/client"

# Builds a multipart/form-data
channel = Channel(String).new(1)
io = IO::Memory.new

HTTP::FormData.build(io) do |formdata|
  channel.send(formdata.content_type)

  formdata.field("user[name]", "Tom")

  File.open("/path/to/file.png") do |file|
    mime = MIME.from_filename(file.path)
    metadata = HTTP::FormData::FileMetadata.new(filename: file.path)
    headers = HTTP::Headers{"Content-Type" => mime}
    formdata.file("user[file]", file, metadata, headers)
  end
end

content_type = channel.receive

# Builds headers
headers = HTTP::Headers{"User-Agent" => "Mozilla/5.0", "Content-Type" => content_type, "Foo" => "bar"}

# Builds cookies
cookies = HTTP::Cookies.new
cookie = HTTP::Cookie.new("foo", "bar")
cookies << cookie

# Adds Cookie headers for the cookies in this collection to the given `HTTP::Headers` instance
cookies.add_request_headers(headers)

# Build a URL encoded HTTP query
params = URI::Params.encode({"author" => "John Doe", "offset" => "20"})

# Make a request
response = HTTP::Client.post(
  URI.new("https", "httpbin.org", path: "/post", query: params),
  headers: headers,
  body: io.to_s
)

puts response.body

Here is an example showing how to do the same with Crest:

require "crest"

response = Crest.post(
  "https://httpbin.org/post",
  form: {"user" => {"name" => "Tom", "file" => File.open("/path/to/file.png")}},
  params: {"author" => "John Doe", "offset" => "20"},
  headers: {"Foo" => "bar"},
  cookies: {"foo" => "bar"},
  user_agent: "Mozilla/5.0"
)

puts response.body

Bonus

Do you remember a curl command from the beginning of this article?
Crest have a .to_curl method to convert Crest::Request object to curl command.

puts response.to_curl
# => curl -X POST https://httpbin.org/post?author=John+Doe&offset=20 -F 'user[name]=Tom' -F 'user[file]=@/home/mama/ruby/crystal/crest/spec/support/fff.png' -H 'Foo: bar' -H 'User-Agent: Mozilla/5.0' -H 'Cookie: foo=bar' -H 'Content-Type: multipart/form-data'

If you have any questions, please let me know. I'd love to get your feedback!

HTTP(S) proxy server in Crystal in less than 50 lines of code

Anton Maminov — Wed, 24 Jun 2020 12:34:04 +0000

The purpose of this tutorial is to implement a proxy server for HTTP and HTTPS in pure Crystal.

Handling of HTTP is a matter of parsing request, passing such request to the destination server, reading response, and passing it back to the client.

HTTPS is different as it'll use a technique called HTTP CONNECT tunneling. At first, the client sends a request using HTTP CONNECT method to set up the tunnel between the client and destination server. When such a tunnel consisting of two TCP connections is ready, the client starts a regular TLS handshake with the destination server to establish a secure connection and later send requests and receive responses.

All we need for that is a built-in Crystal HTTP server and client from HTTP module.

HTTP

To support HTTP we will use a built-in HTTP server and client. The role of the proxy is to handle HTTP requests, pass a request to the desired destination, and send a response back to the client.

HTTP CONNECT tunneling

If a client wants to use HTTPS to talk to the server. The client is aware of using a proxy. A simple HTTP request/response flow cannot be used since the client needs to establish a secure connection with the server (HTTPS). The technique which works is to use the HTTP CONNECT method.

In this mechanism, the client asks a proxy server to forward the TCP connection to the desired destination. The server then proceeds to make the connection on behalf of the client. Once the connection has been established by the server, the proxy server continues to proxy the TCP stream to and from the client. Only the initial connection request is HTTP - after that, the server simply proxies the established TCP connection.

Implementation

Keep in mind the presented code is not a production-grade solution.

require "http"

class HTTP::ProxyHandler
  include HTTP::Handler

  def call(context)
    case context.request.method
    when "CONNECT"
      handle_tunneling(context)
    else
      handle_http(context)
    end
  end

  private def handle_tunneling(context)
    host, port = context.request.resource.split(":", 2)
    upstream = TCPSocket.new(host, port)

    context.response.upgrade do |downstream|
      channel = Channel(Nil).new(2)

      downstream = downstream.as(TCPSocket)
      downstream.sync = true

      spawn do
        transfer(upstream, downstream, channel)
        transfer(downstream, upstream, channel)
      end

      2.times { channel.receive }
    end
  end

  private def transfer(destination, source, channel)
    spawn do
      IO.copy(destination, source)
    rescue ex
      Log.error(exception: ex) { "Unhandled exception on HTTP::ProxyHandler" }
    ensure
      channel.send(nil)
    end
  end

  private def handle_http(context)
    uri = URI.parse(context.request.resource)
    client = HTTP::Client.new(uri)
    response = client.exec(context.request)

    context.response.headers.merge!(response.headers)
    context.response.status_code = response.status_code
    context.response.puts(response.body)
  end
end

proxy_handler = HTTP::ProxyHandler.new
server = HTTP::Server.new([proxy_handler])
address = server.bind_tcp(8080)
puts "Listening on http://#{address}"
server.listen

Our server while getting a request will take one of two paths:

handling HTTP
handling HTTP CONNECT tunneling.

This is done with:

def call(context)
  case context.request.method
  when "CONNECT"
    handle_tunneling(context)
  else
    handle_http(context)
  end
end

Function to handle HTTP — handle_http is self-explanatory so let's focus on handling tunneling.

The first part of handle_tunneling is about setting connection to destination server:

host, port = context.request.resource.split(":", 2)
upstream = TCPSocket.new(host, port)

All the magic happens inside the upgrade handler block:

context.response.upgrade do |downstream|
  ...
end

Once we've two TCP connections (client → proxy, proxy → destination server) we need to set tunnel up:

transfer(upstream, downstream, channel)
transfer(downstream, upstream, channel)

Next, we spawning two fibers and waiting for data are copied in two directions: from the client to the destination server and backward.

A buffered channel of capacity 2 is used to communicate that both fibers ended. So execution goes to the main fiber.

More info about Crystal concurrency you can find in the official documentation

And finally, we run our proxy server.

proxy_handler = HTTP::ProxyHandler.new
server = HTTP::Server.new([proxy_handler])
address = server.bind_tcp(8080)
puts "Listening on http://#{address}"
server.listen

The above code will initialize a server with our proxy handler.
The port of the HTTP server is set by using the method bind_tcp on the object HTTP::Server (the port set to 8080).

For more information, check HTTP::Server documentation

Testing

To test our proxy you can use e.g. Chromium:

chromium-browser --proxy-server=http://localhost:8080

or curl:

curl -Lv --proxy http://localhost:8080 https://httpbin.org/get

Testing the Proxy speeds on my DigitalOcean instance:

Happy Crystalling!