Learn about Ractors and build a mini sidekiq

In this article, you'll learn more about Ractor, and how you can use them to build your own clone of sidekiq (a background processing framework for Ruby).

What is Ractor?

Ruby 3.0 introduced the Ractor class. This is Ruby's Actor-like concurrent abstraction, and its goal is to provide a parallel execution feature of Ruby without thread-safety concerns.

According to wikipedia:

The actor model in computer science is a mathematical model of concurrent computation that treats actor as the universal primitive of concurrent computation.

Actors are able to:

1. Create more actors
2. Receive messages
3. Send messages
4. Take local decisions

Be aware that the Ractor implementation is not stable yet, do NOT use them for production code. See the warning displayed when you use them if you still need to be convinced to not use it (yet) in production:

warning: Ractor is experimental, and the behavior may change in future versions of Ruby! Also there are many implementation issues.

Creating a ractor

Creating a ractor is as simple as using Ractor.new:

ractor = Ractor.new { puts 'Hello Ractor!' }

Receiving a message

There are two ways to receive a message, depending if you have a reference on the ractor sending it.

Use Ractor.receive if you don't know who is sending the message:

message = Ractor.receive

And use Ractor#take if you have a reference on the ractor sending the message:

message = ractor.take

As those method calls are blocking until you receive a message, avoid expecting a message from a ractor that will never send one, or your program is going to be stuck forever.

Also, please be aware that the objects you are sending must be shareable.

Sending a message

If you know your recipient:

ractor.send(message)
# or
ractor << message # `<<` is an alias to `send`

And if you don't:

Ractor.yield(message)

Ractor.yield is blocking as well until some ractor receive your message.

Taking local decision

You can do pretty much anything you want in the block given to Ractor.new. Unless accessing shared objects. An exception will be raised if you try to use a variable defined outside of your block. In order to share a variable between several ractors, you can, for example, use the Ractor::TVar gem.

Another interesting method I'm going to use later on is the Ractor.select(*actors) method. It takes several ractors as an input, and returns the first ractor to send something, and its output:

slow_ractor = Ractor.new { sleep 2; Ractor.yield(:too_late) }
fast_ractor = Ractor.new { Ractor.yield(:fast) }
ractor, output = Ractor.select(slow_ractor, fast_ractor)
# output == :fast && ractor == fast_ractor

Let's build a mini sidekiq!

Please be aware that this crude POC will only allow you to use a pool of 10 ractors to achieve parallel execution of jobs. It won't handle error flow control, statistics, queueing and all the other features that make sidekiq a super useful project.

We'll build a simple design with:

• A WorkerPool, taking care of our pool of ractors
• A Job base class, that all our dedicated jobs will inherit from.

The goal being to allow people to easily implement their own jobs without having to handle all the pool logic.

WorkerPool

class WorkerPool
  attr_reader :ractors

  def initialize
    @ractors = 10.times.map { spawn_worker }
  end

  def spawn_worker
    Ractor.new do
      Ractor.yield(:ready)
      loop { Ractor.yield Job.run(Ractor.receive) }
    end
  end

  def self.run(parameters)
    ractor, _ignored_result =
      Ractor.select(*(@instance ||= new).ractors)
    ractor << parameters
  end
end

There is a trick here. When using Ractor.yield(:ready), we are just making sure that the ractors of the pool have something to send for the initial Ractor.select to work (remember, it's blocking).

Job base class

class Job
  def self.process(*args)
    WorkerPool.run({ class: self, args: args })
  end

  def self.run(hash)
    case hash
      in { class: klass, args: args }
      klass.new.process(*args)
    end
  end
end

Be aware that anything you will provide as argument must be shareable.

Implementing a specific job

Let's say that we'd like to create an asynchronous job that prints something:

class PrintJob < Job
  def process(message)
    puts message
  end
end

Using it asynchronously is now as simple as:

PrintJob.process('Hello World!')

Conclusion

Ractor introduces a new and interesting model for parallel execution in Ruby.

If you found the ractor topic interesting, I suggest you check out these interesting resources:

• https://github.com/ruby/ruby/blob/master/doc/ractor.md
• https://docs.ruby-lang.org/en/3.0.0/Ractor.html

And if you liked this post, check out our awesome weekly tech newsletter. We are regularly sharing top ruby & javascript content!

Photo by Mark Thompson