DEV Community: exAspArk

Introduction to Concurrency Models with Ruby. Part II

exAspArk — Thu, 07 Sep 2017 16:28:58 +0000

Introduction to concurrency models with Ruby. Part II

In the second part of our series we will take a look at more advanced concurrency models such as Actors, Communicating Sequential Processes, Software Transactional Memory and of course Guilds – a new concurrency model which may be implemented in Ruby 3.

If you haven’t read our first post in the series, I’d definitely recommend reading it first. There I described Processes, Threads, GIL, EventMachine and Fibers which I’ll be referring to in this post.

Actors

Actors are concurrency primitives which can send messages to each other, create new Actors and determine how to respond to the next received message. They keep their own private state without sharing it, so they can affect each other only through messages. Since there is no shared state, there is no need for locks.

Do not communicate by sharing memory; instead, share memory by communicating.

Erlang and Scala both implement the Actor model in the language itself. In Ruby, Celluloid is one of the most popular implementations. Under the hood, it runs each Actor in a separate thread and uses fibers for every method invocation to avoid blocking methods while waiting for responses from other Actors.

Here is a basic example of Actors with Celluloid:

# actors.rb
require 'celluloid'
class Universe
  include Celluloid
  def say(msg)
    puts msg
    Celluloid::Actor[:world].say("#{msg} World!")
  end
end
class World
  include Celluloid
  def say(msg)
    puts msg
  end
end
Celluloid::Actor[:world] = World.new
Universe.new.say("Hello")

$ ruby actors.rb
Hello
Hello World!

Pros:

No manual multithreaded programming and no shared memory mean almost deadlock-free synchronization without explicit locks.
Similarly to Erlang, Celluloid makes Actors fault-tolerant, meaning that it’ll try to reboot crashed Actors with Supervisors.
The Actor model is designed to address the problems of distributed programs, so it is great for scaling across multiple machines.

Cons:

Actors may not work if a system needs to use shared state or you need to guarantee a behavior that needs to occur in a specific order.
Debugging can be tricky – imagine following system flow through multiple Actors, or what if some of the Actors mutate the message? Remember that Ruby is not an immutable language, right?
Celluloid allows to build complex concurrent systems much quicker compared to dealing with threads manually. But it does it with a runtime cost (e.g. 5x slower and 8x more memory).
Unfortunately, Ruby implementations are not that great at using distributed Actors across multiple servers. For example, DCell, which uses 0MQ, is still not production ready.

Examples:

Reel – event-based web server, which works with Celluloid-based apps. Uses one Actor per connection. Can be used for streaming or WebSockets.
Celluloid::IO – brings Actors and evented I/O loops together. Unlike EventMachine, it allows to use as many event loops per process as you want by creating multiple Actors.

Communicating Sequential Processes

Communicating Sequential Processes (CSP) is a paradigm which is very similar to the Actor model. It’s also based on message-passing without sharing memory. However, CSP and Actors have these 2 key differences:

Processes in CSP are anonymous, while actors have identities. So, CSP uses explicit channels for message passing, whereas with Actors you send messages directly.
With CSP the sender cannot transmit a message until the receiver is ready to accept it. Actors can send messages asynchronously (e.g. with async calls in Celluloid).

CSP is implemented in such programming languages as Go with goroutines and channels, Clojure with the core.async library and Crystal with fibers and channels. For Ruby, there are a few gems which implement CSP. One of them is the Channel class implemented in concurrent-ruby library:

# csp.rb
require 'concurrent-edge'
array = [1, 2, 3, 4, 5]
channel = Concurrent::Channel.new
Concurrent::Channel.go do
  puts "Go 1 thread: #{Thread.current.object_id}"
  channel.put(array[0..2].sum) # Enumerable#sum from Ruby 2.4
end
Concurrent::Channel.go do
  puts "Go 2 thread: #{Thread.current.object_id}"
  channel.put(array[2..4].sum)
end
puts "Main thread: #{Thread.current.object_id}"
puts channel.take + channel.take

$ ruby csp.rb
Main thread: 70168382536020
Go 2 thread: 70168386894280
Go 1 thread: 70168386894880
18

So, we basically ran 2 operations (sum) in 2 different threads, synchronized the results and calculated the total value in the main thread. Everything is done through the Channel without any explicit locks.

Under the hood, each Channel.go runs in a separate thread from the thread pool, which increases its size automatically if there is no free thread left. In this case, using this model is useful during blocking I/O operations, which release GIL (see the previous post to find more information). On the other hand, core.async in Clojure, for example, uses a limited number of threads and tries to "park" them, but this approach may be a problem during I/O operations which may block out any other work.

Pros:

CSP Channels can only hold maximum one message, which makes it much easier to reason about. While with the Actor model it’s more like having a potentially infinite mailbox with messages.
Communicating Sequential Processes allow you to avoid coupling between the producer and the consumer by using channels; they don’t have to know about each other.
In CSP messages are delivered in the order they were sent.

Clojure may eventually support the actor model for distributed programming, paying the price only when distribution is required, but I think it is quite cumbersome for same-process programming. Rich Hickey

Cons:

CSP is generally used on a single machine, it’s not that great as the Actor model for distributed programming.
In Ruby, most of the implementations don’t use M:N threading model, so each “goroutine” actually uses a Ruby thread, which is equal to an OS thread. This means that Ruby “goroutines” are not so lightweight.
Using CSP with Ruby in not so popular. So, there are no actively developed, stable and battle-tested tools.

Examples:

Agent – another Ruby implementation of CSP. This gem runs each go-block in a separate Ruby thread as well.

Software Transactional Memory

While Actors and CSP are concurrency models which are based on message passing, Software Transactional Memory (STM) is a model which uses shared memory. It’s an alternative to lock-based synchronization. Similarly to DB transactions, these are the main concepts:

Values within a transaction can be changed, but these changes are not visible to others until the transaction is committed.
Errors that happened in transactions abort them and rollback all the changes.
If a transaction can’t be committed due to conflicting changes, it retries until it succeeds.

The concurrent-ruby gem implements TVar, which is based on Clojure’s Refs. Here is an example, which implements money transferring from one bank account to another:

# stm.rb
require 'concurrent'
account1 = Concurrent::TVar.new(100)
account2 = Concurrent::TVar.new(100)
Concurrent::atomically do
  account1.value -= 10
  account2.value += 10
end
puts "Account1: #{account1.value}, Account2: #{account2.value}"

$ ruby stm.rb
Account1: 90, Account2: 110

TVar is an object which contains a single value. Together with atomically they implement data mutation in transactions.

Pros:

Using STM is much simpler compared to lock-based programming. It allows to avoid deadlocks, simplifies reasoning about concurrent systems since you don’t have to think about race conditions.
Much easier to adapt as you don’t need to restructure your code as it’s required with Actors (use models) or CSP (use channels).

Cons:

Since STM relies on transaction rollbacks, you should be able to undo the operation in the transaction at any point in time. In practice, it’s difficult to guarantee if you make I/O operations (e.g. POST HTTP requests).
STM doesn’t scale well with Ruby MRI. Since there is GIL, you can’t utilize more than a single CPU. At the same time, you also can’t use the advantage of running concurrent I/O operations in threads because it’s hard to undo such operations.

Examples:

TVar from concurrent-ruby – implements STM and also contains some benchmarks which compare lock-based implementations and STM in MRI, JRuby and Rubinius.

Guilds

Guild is a new concurrency model proposed for Ruby 3 by Koichi Sasada – a Ruby core developer who designed the current Ruby VM (virtual machine), fibers and GC (garbage collector). These are the main points which lead to creating Guilds:

The new model should be compatible with Ruby 2 and allow better concurrency.
Forcing immutable data structures similar to Elixir may be unacceptably slow since Ruby utilizes many “write” operations. So, it’s better to copy shared mutable objects similarly to Racket (Place) but the copying must be fast for this to be successful.
If it’s necessary to share mutable objects, there should be special data structures similar to Clojure (e.g. STM).

These ideas resulted in the following main concepts of Guilds:

A Guild is a concurrency primitive which may contain multiple threads, which may contain multiple fibers.
Only a Guild-owner can access its mutable objects, so there is no need to use locks.
Guilds can share data by copying objects or by transferring membership (“moving” objects) from one Guild to another.
Immutable objects can be accessed from any Guild by using a reference without copying. E.g. numbers, symbols, true, false, deeply frozen objects.

So, our example of money transferring from one bank account to another with Guilds may look like:

bank = Guild.new do
  accounts = ...
  while acc1, acc2, amount, channel = Guild.default_channel.receive
    accounts[acc1].balance += amount
    accounts[acc2].balance -= amount
    channel.transfer(:finished)
  end
end

channel = Guild::Channel.new
bank.transfer([acc1, acc2, 10, channel])
puts channel.receive
# => :finished

All data about account balances are stored in a single Guild (bank), so, only this Guild is responsible for data mutations which can be requested through channels.

Pros:

No mutable shared data between Guilds means there is no need for locking mechanisms, so there are no deadlocks. Communication between Guilds is designed to be safe.
Guilds encourage using immutable data structures since they are the fastest and the easiest way to share data across multiple Guilds. Start freezing as much data as possible now, for example, by adding # frozen_string_literal: true at the beginning of your files.
Guilds are fully compatible with Ruby 2, meaning that your current code will simply run within a single Guild. You are not required to use immutable data structures or make any changes in your code.
At the same time, Guilds enable better concurrency with MRI. It’ll finally allow us to use multiple CPUs within a single Ruby process.

Cons:

It’s too early to make predictions about performance, but communicating and sharing mutable objects between Guilds will probably have a bigger overhead compared to threads.
Guilds are more complex concurrency primitives because they allow using multiple concurrency models at once. For example: CSP for inter-Guild communication through channels, STM with special data structures for sharing mutable data for better performance, multi-threaded programming within a single Guild, etc.
Even though running multiple Guilds within a single process will be cheaper compared to running multiple processes from a resource usage point of view, Guilds are not so lightweight. They will be heavier than Ruby threads, meaning that you won’t be able to handle, let’s say, tens of thousands of WebSocket connections with just Guilds.

Examples:

There are no examples since Ruby 3 wasn’t released yet. But I see a bright future where developers will start building Guild-friendly tools like web servers, background job processings, etc. Most probably all these tools will allow using hybrid approaches: running multiple processes with multiple Guilds with multiple threads in each. But for now, you can read the original PDF presentation by Koichi Sasada.

Conclusion

There is no silver bullet. Each concurrency model described in the post has its own pros and cons. The CSP model works best on a single machine without deadlocks. The Actor model can easily scale across several machines. STM allows to write concurrent code much simpler. But all these models are not first class citizens in Ruby and can’t be fully adapted from other programming languages; mostly because in Ruby they are all implemented with standard concurrency primitives like threads and fibers. However, there is a chance that Guilds will be released with Ruby 3, which is a big step towards a much better concurrency model!

Originally published on Medium.

Introduction to Concurrency Models with Ruby. Part I

exAspArk — Wed, 23 Aug 2017 15:44:09 +0000

In this first post, I would like to describe the differences between Processes, Threads, what the GIL is, EventMachine and Fibers in Ruby. When to use which of the models, which open-source projects use them, what the pros and cons are.

Processes

Running multiple processes is not actually about concurrency, it’s about parallelism. Although parallelism and concurrency are often confused, they are different things. I like this simple analogy:

Concurrency: having a person juggle many balls with only 1 hand. Regardless of how it seems, the person is only catching / throwing one ball at a time.
Parallelism: is having multiple people juggle their own set of balls simultaneously.

Sequential execution

Imagine, we have a range of numbers, which we need to convert to an array and find an index for the specific element:

# sequential.rb
range = 0...10_000_000
number = 8_888_888
puts range.to_a.index(number)

$ time ruby sequential.rb
8888888
ruby test.rb  0.41s user 0.06s system 95% cpu 0.502 total

Executing this code takes approximately 500ms and utilizes 1 CPU.

Parallel execution

We can rewrite the code above by using multiple parallel processes and splitting the range. With the fork method from the standard Ruby library we can create a child process and execute the code in the block. In the parent process we can wait until all child processes are finished with Process.wait:

# parallel.rb
range1 = 0...5_000_000
range2 = 5_000_000...10_000_000
number = 8_888_888
puts "Parent #{Process.pid}"
fork { puts "Child1 #{Process.pid}: #{range1.to_a.index(number)}" }
fork { puts "Child2 #{Process.pid}: #{range2.to_a.index(number)}" }
Process.wait

$ time ruby parallel.rb
Parent 32771
Child2 32867: 3888888
Child1 32865:
ruby parallel.rb  0.40s user 0.07s system 153% cpu 0.309 total

Because each process works in parallel with just a half of the range, the code above works a bit faster and consumes more than 1 CPU. The process tree during the execution may look like:

# \ - 32771 ruby parallel.rb (parent process)
#  | - 32865 ruby parallel.rb (child process)
#  | - 32867 ruby parallel.rb (child process)

Pros:

Processes don’t share memory, so you can’t mutate date from one process in another. It makes it much easier to code and debug.
Processes in the Ruby MRI are the only way to utilize more than a single-core since there is a GIL (global interpreter lock, find more information below in the post). It may be useful if you’re doing, let’s say, some math calculation.
Forking child processes may help avoid unwanted memory leaks. Once the process finishes, it releases all the resources.

Cons:

Since processes don’t share memory, they use a lot of memory–meaning that running hundreds of processes may be a problem. Note that since Ruby 2.0 fork uses OS Copy-On-Write, which allows processes to share memory as long as it doesn’t have different values.
Processes are slow to create and destroy.
Processes may require inter-process communication. For example, DRb.
Beware of orphan processes (a child process whose parent has finished or terminated) or zombie processes (a child process which completed execution but still occupies space in the process table).

Examples:

Unicorn server – it loads the application, forks the master process to spawn multiple workers which accept HTTP requests.
Resque for background processing – it runs a worker, which executes each job sequentially in a forked child process.

Threads

Even though Ruby uses native OS threads since version 1.9, only one thread can be executing at any given time within a single process, even if you have multiple CPUs. This is due to the fact that MRI has GIL, which also exists in other programming languages such as Python.

Why does the GIL exist?

There are a few reasons, for example:

Avoids race conditions within C extensions, no need to worry about thread-safety.
Easier to implement, no need to make Ruby data structures thread-safe.

Back in 2014, Matz started thinking about gradually removing GIL. Because GIL doesn’t actually guarantee that our Ruby code is thread-safe and doesn’t allow us to use better concurrency.

Race-conditions

Here is a basic example with a race-condition:

# threads.rb
@executed = false
def ensure_executed
  unless @executed
    puts "executing!"
    @executed = true
  end
end
threads = 10.times.map { Thread.new { ensure_executed } }
threads.each(&:join)

$ ruby threads.rb
executing!
executing!

We create 10 threads which execute our method and call join for each of them, so the main thread will be waiting until all other threads are finished. The code printed executing! twice because our threads share the same @executed variable. Our read (unless @executed) and set (@executed = true) operations are not atomic, meaning that once we read the value it could be changed in other threads before we set a new value.

GIL and Blocking I/O

But having GIL, which doesn’t allow to execute multiple threads at once, doesn’t mean that threads can’t be useful. Thread releases GIL when it hits blocking I/O operations such as HTTP requests, DB queries, writing / reading from disk and even sleep:

# sleep.rb
threads = 10.times.map do |i|
  Thread.new { sleep 1 }
end
threads.each(&:join)

$ time ruby sleep.rb
ruby sleep.rb  0.08s user 0.03s system 9% cpu 1.130 total

As you can see, all 10 threads slept for 1 second and finished almost at the same time. When one thread hit sleep, it passed the execution to another thread without blocking GIL.

Pros:

Uses less memory than processes; it’s possible to run thousands of threads. They are also fast to create and destroy.
Threads are useful when there are slow blocking I/O operations.
Can access the memory area from other threads if necessary.

Cons:

Requires very careful synchronization to avoid race-conditions, usually by using locking primitives, which sometimes may lead to deadlocks. All this makes it quite difficult to write, test and debug thread-safe code.
With threads you have to make sure that not only your code is thread-safe, but that any dependencies you’re using are also thread-safe.
The more threads you spawn, the more time and resources they’ll be spending by switching the context and spending less time doing the actual job.

Examples:

Puma server – allows to use multiple threads in each process (clustered mode). Similarly to Unicorn it preloads the app and forks the master process, where each child process has its own thread pool. Threads work fine in most cases because each HTTP request can be handled in a separate thread and we don’t share a lot of resources between the requests.
Sidekiq for background processing – runs a single process with 25 threads by default. Each thread processes one job at a time.

EventMachine

EventMachine (aka EM) is a gem which is written in C++ and Ruby. It provides event-driven I/O using the Reactor pattern and can basically make your Ruby code looks like Node.js :) Under the hood EM uses Linux select() during its run through the event loop to check for new inputs on file descriptors.

One common reason to use EventMachine is the case when you have a lot of I/O operations and you don’t want to deal with threads manually. Manually handling threads can be difficult or often too expensive from a resource usage point of view. With EM you can handle multiple HTTP requests with a single thread by default.

# em.rb
EM.run do
  EM.add_timer(1) do
    puts 'sleeping...'
    EM.system('sleep 1') { puts "woke up!" }
    puts 'continuing...'
  end
  EM.add_timer(3) { EM.stop }
end

$ ruby em.rb
sleeping...
continuing...
woke up!

The example above shows how to run asynchronous code by executing EM.system (I/O operation) and passing a block as a callback, which will be executed once the system command has finished.

Pros:

It’s possible to achieve great performance for slow networked apps such as web servers and proxiesÂ withÂ aÂ singleÂ thread.
It allows you to avoid complex multithreaded programming, the disadvantages of which were described above.

Cons:

Every I/O operation should support EM asynchrony. This means that you should use specific versions of system, DB adapter, HTTP client, etc. which can result in monkey-patched versions, lack of support and limited options.
Work done within the main thread per event-loop tick should be small. Also, it’s possible to use Defer, which executes the code in separate threads from the thread pool, however, it may lead to the multithreaded problems discussed earlier.
Hard to program complex systems because of the error handling and callbacks. Callback Hell is also possible in Ruby, but it can be prevented with Fibers, see below.
EventMachine itself is a huge dependency: 17K LOC (lines of code) in Ruby and 10K LOC in C++.

Examples:

Goliath – single threaded asynchronous server.
AMQP – RabbitMQ client. However, creators of the gem suggest using the non-EM-based version Bunny. Note that migrating tools to EM-less implementations is a general trend. For example, creators of ActionCable decided to use low-level nio4r, creator of sinatra-synchrony rewrote it with Celluloid, etc.

Fibers

Fibers are light weight primitives in the Ruby standard library which can be paused, resumed and scheduled manually. They are pretty much the same as ES6 Generators if you’re familiar with JavaScript (we also wrote a post about Generators and Redux-Saga). It’s possible to run tens of thousands of Fibers within a single thread.

Often, Fibers are used with EventMachine to avoid callbacks and make code look synchronous. So, the following code:

EventMachine.run do
  page = EM::HttpRequest.new('https://google.ca/').get

  page.errback { puts "Google is down" }
  page.callback {
    url = 'https://google.ca/search?q=universe.com'
    about = EM::HttpRequest.new(url).get

    about.errback  { ... }
    about.callback { ... }
  }
end

Can be rewritten like:

EventMachine.run do
  Fiber.new {
    page = http_get('http://www.google.com/')
    if page.response_header.status == 200
      about = http_get('https://google.ca/search?q=universe.com')
      # ...
    else
      puts "Google is down"
    end
  }.resume
end

def http_get(url)
  current_fiber = Fiber.current
  http = EM::HttpRequest.new(url).get
  http.callback { current_fiber.resume(http) }
  http.errback  { current_fiber.resume(http) }
  Fiber.yield
end

So, basically, Fiber#yield returns control back to the context that resumed the Fiber and returns the value which was passed to Field#resume.

Pros:

Fibers allow you to simplify asynchronous code by replacing nested callbacks.

Cons:

Don’t really solve concurrency problems.
They are rarely used directly in application-level code.

Examples:

em-synchrony – a library, written by Ilya Grigorik, a performance engineer at Google, which integrates EventMachine with Fibers for different clients such as MySQL2, Mongo, Memcached, etc.

Conclusion

There is no silver bullet, so choose a concurrency model depending on your needs. For example, need to run CPU and memory intensive code and have enough resources – use processes. Have to execute multiple I/O operations such as HTTP requests – use threads. Need to scale up to the maximum throughput – use EventMachine.

In the second part of this series we will take a look at such concurrency models as Actors (Erlang, Scala), Communicating Sequential Processes (Go, Crystal), Software Transactional Memory (Clojure) and of course Guilds – a new concurrency model which may be implemented in Ruby 3. Stay tuned!

Originally published on Medium.

Batching – A powerful way to solve N+1 queries every Rubyist should know

exAspArk — Wed, 16 Aug 2017 15:45:09 +0000

In this post, I’m going to tell you about batching as a technique to help avoid N+1 queries, existing battle-tested tools like Haskell Haxl and JavaScript DataLoader, and how similar approaches can be used in any Ruby program.

What are N+1 queries?

First, let’s find out what N+1 queries are and why they are called that. Imagine, we have 2 SQL tables: users and posts. If we run the following code by using ActiveRecord models:

posts = Post.where(id: [1, 2, 3])
# SELECT * FROM posts WHERE id IN (1, 2, 3)

users = posts.map { |post| post.user }
# SELECT * FROM users WHERE id = 1
# SELECT * FROM users WHERE id = 2
# SELECT * FROM users WHERE id = 3

This makes 1 query to select all posts, then N queries to select users for each post. So, the code produces 1+N queries. Because changing the order of the addends does not change the sum, it’s common to call it N+1 queries.

How do people usually solve N+1 queries?

Generally, there are 2 popular ways to solve the problem of N+1 queries in the Ruby world:

Eager loading data in models

posts = Post.where(id: [1, 2, 3]).includes(:user)
# SELECT * FROM posts WHERE id IN (1, 2, 3)
# SELECT * FROM users WHERE id IN (1, 2, 3)

users = posts.map { |post| post.user }

It preloads the specified associations under the hood, which makes it easy to use. But ORM can’t always help. For example, in a case when we need to load data from different sources such as another database.

Preloading data and passing it through arguments

class Post < ApplicationRecord
  def rating(like_count, angry_count)
    like_count * 2 - angry_count
  end
end

posts = Post.where(id: [1, 2, 3])
# SELECT * FROM posts WHERE id IN (1, 2, 3)

post_emoticons = Emoticon.where(post_id: posts.map(&:id))

like_count_by_post_id = post_emoticons.like.group(:post_id).count
angry_count_by_post_id = post_emoticons.angry.group(:post_id).count
# SELECT COUNT(*) FROM emoticons WHERE name = 'like' AND post_id IN (1, 2, 3) GROUP BY post_id
# SELECT COUNT(*) FROM emoticons WHERE name = 'angry' AND post_id IN (1, 2, 3) GROUP BY post_id

posts.map do |post|
  post.rating(
    like_count_by_post_id[post.id],
    angry_count_by_post_id[post.id]
  )
end

This method is very flexible and can work when the simpler method of using includes can’t. And it may also be more memory efficient – in our example above we don’t load all emoticons to calculate a rating for each post. Instead, our database can do all the hard work and return just a count for each post, which is passed as an argument. However, passing the data through arguments can be complicated, especially when there are several layers below (e.g. load Emoticons, pass through Users to Posts).

Both of these approaches can help us to avoid N+1 queries. But the problem is that we should know in advance on the top level which data we need to preload. And we will preload it every time, even if it’s not necessary. For example, with GraphQL these approaches don’t work since with GraphQL we can’t predict which fields users are going to ask in a query. Is there any other way to avoid N+1 queries? Yes, it’s called batching.

What is batching?

Batching isn’t a new way to solve N+1 queries. Facebook released the Haskel Haxl library in 2014, but the technique has been used long before. It uses such concepts as Monad, Applicative and Functor. I won’t get into explaining these ideas as it deserves a separate post (would you be interested in learning more about functional programming in Ruby?).

The idea of batching was also implemented in other programming languages. One of the most well-known libraries is JavaScript DataLoader, which became very popular with the rise of GraphQL. Here is a great video about its source code by Lee Byron, an engineer at Facebook. The code is pretty straight forward and contains just 300 lines.

These are the general steps for batching:

Passing an item to load in any part of the app.
Loading and caching values for the passed items in batch.
Getting the loaded value where the item was passed.

The main advantage of using this technique is that batching is isolated. It allows to load data where and when it’s needed.

Here is a basic example of using JavaScript DataLoader:

var batch = (userIds) => ...;
var loader = new DataLoader(userIds => batch(userIds));

// “load” schedules a job to dispatch a queue with
// Node.js “process.nextTick” and returns a promise

loader.load(userId1).then(user1 => console.log(user1));
loader.load(userId2).then(user2 => console.log(user2));
loader.load(userId3).then(user3 => console.log(user3));

First, we are creating a loader with a function which accepts all the collected items to load (userIds). These items are passed to our batch function which loads all users at once. Then we can call the loader.load function which returns a promise with a loaded value (user). OK, but what about Ruby?

Batching in Ruby

At Universe, we have monthly hackathons when everyone is free to experiment with any ideas and technologies such as Ethereum, Elixir, Progressive Web App, etc. During my last hackathon, which I won in 1 of the nominations, I was learning about existing techniques to avoid N+1 queries in GraphQL and building a tool to transform GraphQL query to MongoDB Aggregation Pipeline for batching. Check out our previous post which describes how to use Aggregation Pipeline to “join” different collections, filter them, serialize and so on.

At the same time, while we’re migrating to GraphQL, we’re still supporting our RESTful APIs. Usually, N+1 DB queries and HTTP requests are the main problems which cause bottlenecks in our applications as we continue to scale our platform.

That is why we decided to create a new tool which will allow us to solve N+1 queries in our existing RESTful APIs as well as in GraphQL. A simple tool which every Ruby developer will be able to understand and use. It’s called BatchLoader.

Laziness

class Post < ApplicationRecord
  belongs_to :user

  def user_lazy
    # something cool with BatchLoader
  end
end

posts = Post.where(id: [1, 2, 3])
# SELECT * FROM posts WHERE id IN (1, 2, 3)

users_lazy = posts.map { |post| post.user_lazy }

BatchLoader.sync!(users_lazy)
# SELECT * FROM users WHERE id IN (1, 2, 3)

BatchLoader doesn’t try to mimic implementations in other programming languages which have an asynchronous nature. So, it doesn’t use any extra primitives such as Promises. There is no reason to use them in Ruby unless you’re using something like EventMachine.

Instead, it uses the idea of “lazy objects”, which are used in Ruby standard library. For example, “lazy arrays” – they allow to manipulate with elements and resolve them at the end when it’s necessary:

range = 1..Float::INFINITY

values_lazy = range.lazy.map { |i| i * i }.take(10)

values_lazy.force
# => [1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

As you can see, the 2 code blocks above have the same pattern:

Collect lazy objects.
Resolve them at the end.

Batching

Now let’s have a closer look at Post#user_lazy, which returns a lazy BatchLoader instance:

# app/models/post.rb
def user_lazy
  BatchLoader.for(user_id).batch do |user_ids|
    User.where(id: user_ids)
  end
end

BatchLoader.for accepts an item (user_id) which should be collected and used for batching later. Then we call the batch method where we pass a block which will use all the collected items (user_ids). Inside the block, we execute a batch query for our items (User.where).

JavaScript DataLoader maps the passed item and loaded value implicitly. But it relies on these 2 constraints:

The array of passed items (user_ids) must be the same length as the array of loaded values (users). This usually means that we have to add nils in the array for absent values.
Each index in the array of passed items must correspond to the same index in the array of loaded values. This usually means that we should sort the loaded values.

BatchLoader, in this case, provides a load method which can be simply called to map a passed item (user_id) to loaded value (user):

# app/models/post.rb
def user_lazy
  BatchLoader.for(user_id).batch do |user_ids, batch_loader|
    User.where(id: user_ids).each { |u| batch_loader.load(u.id, u) }
  end
end

RESTful API example

Now imagine we have a regular Rails application with N+1 HTTP requests:

# app/models/post.rb
class Post < ApplicationRecord
  def rating
    HttpClient.request(:get, "https://example.com/ratings/#{id}")
  end
end

# app/controllers/posts_controller.rb
class PostsController < ApplicationController
  def index
    posts = Post.limit(10)
    serialized_posts = posts.map do |post| 
      {id: post.id, rating: post.rating} # <== N+1 HTTP requests
      end

    render json: serialized_posts
  end
end

We can batch the requests with a gem called parallel by executing all HTTP requests concurrently in threads. Thankfully, MRI releases GIL (global interpreter lock) when a thread hits blocking I/O – HTTP request in our case.

# app/models/post.rb
def rating_lazy
  BatchLoader.for(post).batch do |posts, batch_loader|
    Parallel.each(posts, in_threads: 10) do |post|
      batch_loader.load(post, post.rating)
    end
  end
end

# app/controllers/posts_controller.rb
class PostsController < ApplicationController
  def index
    posts = Post.limit(10)
    serialized_posts = posts.map do |post| 
      {id: post.id, rating: post.lazy_rating}
    end

    render json: BatchLoader.sync!(serialized_posts)
  end
end

Thread-safety

The example with concurrent HTTP requests in threads will work only if HttpClient is thread-safe. BatchLoader#load is thread-safe out of the box, so it doesn’t need any extra dependencies.

GraphQL example

Batching is particularly useful with GraphQL. Using such techniques as preloading data in advance to avoid N+1 queries can be very complicated since a user can ask for any available fields in a query. Let’s take a look at the simple graphql-ruby schema example:

Schema = GraphQL::Schema.define do
  query QueryType
end

QueryType = GraphQL::ObjectType.define do
  name "Query"
  field :posts, !types[PostType], resolve: ->(obj, args, ctx) do
    Post.all
  end
end

PostType = GraphQL::ObjectType.define do
  name "Post"
  field :user, !UserType, resolve: ->(post, args, ctx) do
    post.user # <== N+1 queries
  end
end

UserType = GraphQL::ObjectType.define do
  name "User"
  field :name, !types.String
end

If we want to execute a simple query like the following, we will get N+1 queries for each post.user:

query = "
{
  posts {
    user {
      name
    }
  }
}
"
Schema.execute(query)
# SELECT * FROM posts WHERE id IN (1, 2, 3)
# SELECT * FROM users WHERE id = 1
# SELECT * FROM users WHERE id = 2
# SELECT * FROM users WHERE id = 3

To avoid this problem, all we have to do is to change the resolver to use BatchLoader:

PostType = GraphQL::ObjectType.define do
  name "Post"
  field :user, !UserType, resolve: ->(post, args, ctx) do
    BatchLoader.for(post.user_id).batch do |ids, batch_loader|
      User.where(id: ids).each { |u| batch_loader.load(u.id, u) }
    end
  end
end

And setup GraphQL with the built-in lazy_resolve method:

Schema = GraphQL::Schema.define do
  query QueryType
  lazy_resolve BatchLoader, :sync
end

That’s it. GraphQL lazy_resolve will basically call a resolve lambda on the field. If it returns an instance of a lazy BatchLoader, it’ll call BatchLoader#sync later to get the actual loaded value automatically.

Caching

BatchLoader also provides a caching mechanism out of the box. So, it won’t make queries for already loaded values. For example:

def user_lazy(id)
  BatchLoader.for(id).batch do |ids, batch_loader|
    User.where(id: ids).each { |u| batch_loader.load(u.id, u) }
  end
end

user_lazy(1)      # no request
# => <#BatchLoader>

user_lazy(1).sync # SELECT * FROM users WHERE id IN (1)
# => <#User>

user_lazy(1).sync # no request
# => <#User>

Conclusion

Overall, batching is a powerful technique to avoid N+1 queries. I believe that every Ruby developer should know about it, and not just people who use GraphQL or other programming languages. It will allow you to decouple unrelated parts of your application and load your data where and when it’s needed in batch without sacrificing the performance.

At Universe, we’re using BatchLoader on production for both RESTful API and GraphQL by sharing the same code. For more information, check out the README and the source code, which has just 150 lines.

Originally published on Medium.