DEV Community: Renata Marques

Mastering Ruby: Sequel and ActiveRecord, an overview

Renata Marques — Tue, 21 Nov 2023 13:54:12 +0000

Sequel and ActiveRecord are object-relational mapping for ruby-based applications, they have similarities and disparities, and this article will guide you to choose which one best suits your needs.

If you are not familiar with the term ORM, here is a short explanation: ORMs provide a set of tools and conventions for mapping database tables and rows to corresponding objects and their attributes in the application code. This simplifies database interactions, eliminates the need to write raw SQL queries (but you can if you want), and allows developers to work with data using familiar programming paradigms.

What is ActiveRecord?

ActiveRecord is an Object-Relational Mapping (ORM) library that is an integral part of the Ruby on Rails framework. It provides an abstraction layer that allows you to interact with a database using Ruby objects and methods rather than writing raw SQL queries. ActiveRecord simplifies database interactions, data validation, and the handling of associations between different types of data.

How can I use it?

Here's how you can use ActiveRecord in a Ruby-based application, particularly within the context of Ruby on Rails:

Setup and Configuration:
In a Rails application, ActiveRecord is included by default. There's no need to install it separately.

Configure your database connection in the config/database.yml file. Specify the database adapter, host, database name, user, password, and other connection details.

Creating Models:
Create Ruby classes that inherit from ActiveRecord::Base. Each model class typically corresponds to a database table.
Define attributes as instance variables in the model class to represent table columns.
Establish associations (e.g., has_many, belongs_to, has_and_belongs_to_many) between model classes to define relationships in the database.

Migrations:
Use migrations to manage your database schema. Migrations allow you to create, modify, and delete database tables and columns.
Migrations provide a version control-like system for your database structure, enabling you to keep track of changes over time.

CRUD Operations:
Create: Use the create method on your model class to insert new records into the database.
Read: Use methods like find, where, and all to retrieve records from the database.
Update: Use the update method or combination of find and save to modify existing records.
Delete: Use the destroy method to remove records from the database.

Validation:
Define validation rules using ActiveRecord's built-in validation methods. These ensure that the data entering the database meets specific criteria.
Common validations include presence, length, uniqueness, and more.

Callbacks:
Use callbacks to execute code at specific points in the lifecycle of a model instance. Examples include before_save, after_create, and before_destroy.

Associations:
Leverage ActiveRecord's association methods to establish relationships between different model classes. This allows you to navigate and manipulate related data.

Query Interface:
Use ActiveRecord's query interface to construct and execute complex database queries using a fluent and Ruby-like syntax.
Methods like where, joins, group, and order can be combined to build intricate queries.

Console and Rails Console:
Use the Rails console (rails console) to interact with your models and database in a live environment.
Test your queries, create records, and perform various operations directly from the console.

Error Handling:
ActiveRecord provides exceptions and error messages when validation or database-related errors occur, helping you diagnose and troubleshoot issues.

With AR you can also write custom SQL queries.

You can check the documentation: https://guides.rubyonrails.org/active_record_basics.html.

What is Sequel?

Sequel is a popular Object-Relational Mapping (ORM) library for Ruby that provides a simple and flexible way to interact with databases. It offers an alternative to ActiveRecord, the default ORM in Ruby on Rails. Sequel is known for its lightweight design, extensive feature set, and focus on providing a more SQL-centric approach.

How can I use it?

Here's how you can use Sequel in your Ruby-based application:
Installation: Add the Sequel gem to your Gemfile and run bundle install.

Database Configuration: Configure your database connection in your application. Sequel supports a wide range of database systems, including PostgreSQL, MySQL, SQLite, and more. You'll need to specify the appropriate adapter, host, database name, user, password, and any other relevant connection options.

Model Definitions: Define your models by creating Ruby classes that inherit from Sequel::Model. Each model class should represent a table in your database. Define associations, validations, and other model-specific configurations using Sequel's API.

Queries: Use Sequel's query methods to interact with the database. Sequel provides a wide range of methods to build complex queries, including filtering, sorting, joining tables, and more.

Creating Records: Instantiate your model classes to create new records in the database. Sequel provides an intuitive syntax to insert data.

Retrieving Records: Use the model class's methods to retrieve records from the database. You can chain query methods to build complex queries.

Updating and Deleting Records: Use Sequel's methods to update and delete records from the database.

Associations: Define and work with associations between models using Sequel's association methods. These include one-to-one, one-to-many, and many-to-many associations.

Migrations: Migrations are optional, you don’t have to use them. However, you can use Sequel's migration system to manage changes to your database schema over time.

Hooks and callbacks: Implement hooks and callbacks to perform actions before or after certain events, such as saving records or validating data.

Error Handling: Sequel provides error handling mechanisms to catch and manage exceptions that might occur during database operations.

Custom SQL: While Sequel offers a wide range of high-level query methods, you can also write custom SQL queries when needed.

You can check the syntax and more information about the Sequel ORM: https://github.com/jeremyevans/sequel.

ActiveRecord vs. Sequel

Choosing between Sequel and ActiveRecord depends on your project's requirements and preferences.

Here are some scenarios where using Sequel might be more suitable than ActiveRecord:
Choosing between Sequel and ActiveRecord depends on your project's requirements and preferences. Here are some scenarios where using Sequel might be more suitable than ActiveRecord:

Performance and Control: Sequel is known for its lightweight design and can offer better performance in certain scenarios. If you need more control over how queries are executed and want to optimize performance, Sequel's flexibility can be an advantage.

Postgres Features: If you are using Postgres, Sequel employs the PG extended query protocol, making it significantly faster than ActiveRecord and reducing server-side resource usage. You can learn more about it (here)[https://www.postgresql.org/docs/current/protocol-flow.html#PROTOCOL-FLOW-EXT-QUERY]. Unlike Sequel, ActiveRecord doesn't fully leverage the potential of PostgreSQL.

SQL-Centric Approach: Sequel has a powerful SQL DSL. If you're comfortable working with a more SQL-oriented approach to database interactions, Sequel's API aligns closely with SQL syntax. This can be advantageous when you need fine-grained control over your queries.
Calling functions:

# Using Sequel

# defining the function on the database
DB.run(<<~SQL)
  CREATE FUNCTION add_numbers(a integer, b integer)
  RETURNS integer AS $$
  BEGIN
      RETURN a + b;
  END; $$
  LANGUAGE plpgsql;
SQL

Calling the function somewhere in the app.

Sequel.function(:add_numbers, 1, 2)
DB.select(f).first
# => {:add_numbers=>3}

# Using ActiveRecord

# defining the function on the database
execute <<~SQL
  CREATE OR REPLACE FUNCTION add_numbers(a integer, b integer)
  RETURNS integer AS $$
  BEGIN
    RETURN a + b;
  END;
  $$ LANGUAGE plpgsql;
SQL

Calling the function somewhere in the app.

# Call the SQL function using a raw SQL query
result = ActiveRecord::Base.connection.exec_query("SELECT add_numbers(1, 2) AS add_numbers")

# Access the result
add_numbers_result = result.first["add_numbers"]

This shows Sequel has better Ruby DSLs for calling functions, rather than having to go down and write raw SQL to call functions.

Advanced Features: Sequel also provides advanced features like subqueries, window functions, and CTEs (Common Table Expressions).
Subqueries example:

# Using Sequel

# Find users with the highest salary in each department
subquery = DB[:salaries].select_group(:department_id).select_append{max(salary).as(max_salary)}
result = DB[:salaries].join(subquery, [:department_id, :max_salary])

# Using ActiveRecord

# Find users with the highest salary in each department
subquery = Salary.select("department_id, MAX(salary) AS max_salary").group(:department_id)
result = Salary.joins("INNER JOIN (#{subquery.to_sql}) max_salaries ON salaries.department_id = max_salaries.department_id AND salaries.salary = max_salaries.max_salary")

Window functions example:

# Sequel

# Calculate the average salary for each employee
result = DB[:salaries].select_group(:employee_id).select_append{avg(salary).over(partition: :employee_id).as(average_salary)}

# ActiveRecord

# Calculate the average salary for each employee
result = Salary.select("employee_id, AVG(salary) OVER (PARTITION BY employee_id) AS average_salary")

CTEs example:

# Sequel

# Find employees with salaries above the department's average salary using a CTE
cte = DB[:salaries].select_group(:department_id).select_append{avg(salary).as(average_salary)}
result = DB[:salaries].with(:cte, cte).where{salary > cte[:average_salary]}

# ActiveRecord

# Find employees with salaries above the department's average salary using a subquery
subquery = Salary.select("department_id, AVG(salary) AS average_salary").group(:department_id)
result = Salary.joins("INNER JOIN (#{subquery.to_sql}) avg_salaries ON salaries.department_id = avg_salaries.department_id").where("salaries.salary > avg_salaries.average_salary")

Here we can see how challenging is to achieve the same thing using ActiveRecord's query interface.

Using Hanami or ROM: Hanami and ROM use Sequel under the hood, which is a powerful source and an alternative for Ruby on Rails.

Methods at build time: Sequel provides a metaprogramming feature that allows you to define methods at the time you define your model class. These methods are shared among all instances of the class. It can lead to more efficient memory usage, as the methods are shared among instances. It can also make your code more organized, as methods are defined alongside the class.

Complex Queries: When dealing with complex database queries, joins, subqueries, and advanced filtering, Sequel's expressive query-building methods can make it easier to construct intricate queries without sacrificing readability.

Custom Data Types: If your application works with custom or non-standard data types that require more control over serialization and deserialization, Sequel's custom-type support can be beneficial.

Legacy Databases: If your application interacts with a legacy database schema that doesn't adhere to Rails conventions, Sequel's ability to work with non-standard column and table names can be advantageous.

Less Magic: Sequel emphasizes explicitness and minimizes "magic" behaviors. If you prefer to have a clearer understanding of what happens behind the scenes in your database interactions, Sequel might be a better fit.

Lean Applications: For applications where you want to keep the overhead of the ORM to a minimum, Sequel's modular design and lack of excessive abstractions can be beneficial.

Small Projects: Sequel's straightforward and focused API can be more suitable for smaller projects where you don't need all the features and complexity of ActiveRecord.

Here are some scenarios where using ActiveRecord might be more suitable than Sequel:
You're building a Ruby on Rails application: ActiveRecord is tightly integrated into the Ruby on Rails framework. If you're building a Rails application, using ActiveRecord provides a cohesive and familiar development experience.

Convention Over Configuration: ActiveRecord follows Rails' convention over configuration principle. It comes with sensible defaults and a standardized approach to database interactions, which can speed up development.

Fast Development: ActiveRecord's ease of use, automagic associations, and built-in query methods can help you quickly build applications with less boilerplate code.

Community and Ecosystem: ActiveRecord is widely used, which means you'll find ample documentation, tutorials, and community support. If you're new to Ruby on Rails, the abundant resources can be a significant advantage.

ORM Integration: If your project involves features closely integrated with Rails, such as form helpers, view scaffolding, and other framework-specific components, ActiveRecord's seamless integration is beneficial.

It's important to note that ActiveRecord is the default ORM for Ruby on Rails, which provides a cohesive development experience for most Rails projects. ActiveRecord abstracts away much of the complexity of database interactions and integrates seamlessly with Rails conventions.

Disadvantages of both:
Using views: By default, database views are treated as read-only when using ActiveRecord. This means that you can query data from views just like you would from tables, but you cannot perform direct write operations (insert, update, or delete) on views through ActiveRecord. Sequel, like ActiveRecord, treats views as read-only by default. Sequel models associated with views are typically used for querying data, not for performing direct write operations.

Summary

Deciding between Sequel and ActiveRecord depends on your specific needs, preferences, and the nature of your project. Ultimately, the decision between Sequel and ActiveRecord should be based on your familiarity with SQL, your project's requirements, your team's preferences, and the specific trade-offs you're willing to make. If you value control, performance, flexibility, and a more SQL-centric approach, Sequel could be a strong choice. If you prioritize convention, ease of use, and seamless integration with Rails, ActiveRecord might be a better fit.

Mastering Ruby: Common Performance Issues

Renata Marques — Tue, 21 Nov 2023 13:48:39 +0000

Why "Ruby is slow"? Well, some people say Ruby is slow based on:

Ruby is an interpreted language, and interpreted languages tend to be slower than compiled ones.
Ruby uses garbage collection (While Ruby uses garbage collection, it's worth noting that languages like C#, which also utilize garbage collection, demonstrate significantly superior performance compared to Ruby, Python, PHP, and similar languages.)
Ruby method calls are slower than other programming languages (although, because of duck typing, they are arguably faster than in strongly typed interpreted languages).
Ruby does not support **true multithreading** (with the exception of JRuby).

This is a list of problems that cause slowdowns in applications using Ruby and Ruby on Rails, the most popular Ruby-based web framework.

Slowness
Memory leaking
N+1 queries
Slow queries
API payloads

Slowness

Lack of Indexing: If your database tables are not properly indexed, queries can take longer to execute, especially for large datasets.

Inefficient Algorithms: Poorly optimized algorithms can lead to slow execution times, especially for large datasets. Choosing the right data structures and algorithms can significantly improve performance.

Ruby Version: Older versions of Ruby may lack performance optimizations and improvements available in newer versions.

Framework and Library Choices: Some Ruby gems and libraries may not be optimized for performance, leading to slower execution times. Choose well-maintained and widely-used libraries when possible.

Solutions for slowness

I strongly suggest measuring your application's performance before you change anything and then again after each change.

Optimize Ruby code; use the built-in classes and methods when available rather than developing your own.

Reduce nested if/else, loops, and hashes if possible; all of these operations are really expensive, and sometimes just refactoring your code can reduce the need.

Don’t be afraid of using features of your database; for example, define stored procedures and functions, knowing that you can use them by communicating directly with the database through driver calls rather than ActiveRecord high-level methods. Implementing this technique can greatly enhance the performance of a data-bound Rails application.

It involves storing content generated during the request-response cycle and reusing it when responding to similar requests, effectively boosting the application's speed. When you repeatedly request identical data throughout the processing of a single request and class-level caching isn't feasible due to data dependency on request parameters, consider caching the data to prevent redundant computations.

Memoization involves caching the result of a method so subsequent calls to the method return the cached result, saving processing time. It is a type of caching focused on storing function return values. However, keep in mind that Memoization should be avoided in some cases, for example, when a function is going to change over time and your business logic relies on the latest value.

You can learn more about memoization here: A guide to memoization in Ruby.

Detecting and avoiding code smells

Avoiding code duplication, unused variables, or method arguments, and detecting code smells with the use of a static code analyzer such as [Rubocop](https://github.com/rubocop/rubocop) and RubyCritic](https://rubygems.org/gems/rubycritic/versions/2.9.1).

Use HTML in your view templates and avoid excessive use of helpers. When utilizing form helpers, be mindful that they add an extra layer of processing.

Keep your Ruby version updated.

Manage your controllers wisely; filters can be costly, so avoid overusing them. Additionally, be mindful of using excessive instance variables that your view doesn't genuinely need; they can significantly impact performance.

Avoid unnecessary gems. Only include gems that are genuinely required for your application. This reduces potential conflicts and keeps your application leaner.

Keep your gems up to date to benefit from bug fixes, security patches, and new features. However, test thoroughly after updates to avoid compatibility issues.

N+1 Query

In the context of database queries, an N+1 query is a situation where, when retrieving a list of records (N), each record triggers an additional query (+1) to the database for related data. This can lead to performance issues as the number of queries increases with the number of records retrieved, resulting in an inefficient use of resources and slower response times.

Let's illustrate this with an example in Ruby using ActiveRecord, a popular Object-Relational Mapping (ORM) library:

Suppose we have two models, User and Post, where each user has many posts, and the models are defined as follows:



# app/models/user.rb
class User < ApplicationRecord
  has_many :posts
end

# app/models/post.rb
class Post < ApplicationRecord
  belongs_to :user
end

Now, let's say we want to retrieve all users and their posts:



# N+1 query example
@users = User.all

@users.each do |user|
  puts "User: #{user.name}"
  puts "Posts:"
  user.posts.each do |post|
    puts post.title
  end
end

In this example, the User.all query retrieves all users, but for each user, there is an additional query executed to fetch their associated posts. This leads to N+1 queries: one initial query to fetch all users and N additional queries (one per user) to fetch their posts.

The best way to find n+1 queries is by reading your operation’s log. Then, you can see a record of every database query being made as part of each request and look for cases that might indicate an n+1 query that can be optimized.

Generally, an n+1 query will look like numerous analogous queries being executed one after the other:



SELECT “posts”.* FROM “posts” LIMIT 3

SELECT "users".* FROM "users" WHERE "users"."id" = 1 LIMIT 1

SELECT "users".* FROM "users" WHERE "users"."id" = 2 LIMIT 1

SELECT "users".* FROM "users" WHERE "users"."id" = 3 LIMIT 1

Possible solutions to address the N+1 query issue

Eager loading is a technique where you fetch the associated records in a single query, reducing the number of database trips.



# Eager loading solution using `includes`
@users = User.includes(:posts).all

@users.each do |user|
  puts "User: #{user.name}"
  puts "Posts:"
  user.posts.each do |post|
    puts post.title
  end
end

Joins: You can use a SQL JOIN to retrieve both users and their posts in a single query.



# Join solution
@users = User.joins(:posts).select('users.*, posts.title as post_title')

@users.each do |user|
  puts "User: #{user.name}"
  puts "Post: #{user.post_title}"
end

Use includes with nested associations: If you have multiple levels of associations, you can use nested includes to avoid multiple queries.



# Nested includes solution
@users = User.includes(posts: :comments).all

@users.each do |user|
  puts "User: #{user.name}"
  puts "Posts:"
  user.posts.each do |post|
    puts "Title: #{post.title}"
    puts "Comments:"
    post.comments.each do |comment|
      puts comment.text
    end
  end
end

By using these solutions, you can avoid N+1 queries and improve the performance of your application when dealing with associated data.

So, N+1 queries have been a consideration and concern in Ruby on Rails applications for many versions, and the best practices to avoid them have been part of the Rails development community's guidelines. Always ensure you are using the most recent version of Ruby on Rails to take advantage of the latest performance improvements and optimizations. The Bullet or Prosopite gems can give a hand here by telling when to use eager loading and when not, and by adding a counter cache.

Memory Leaks

Ruby's garbage collector should automatically clean up unused objects, but memory leaks can occur if objects are not properly deallocated. Frequent garbage collection can lead to slower execution times.

Here you can learn more how to track memory leaks.

Memory leaks can be very unproductive and difficult to fix, and you will probably spend more time trying to fix them than actually building code. The best way to get rid of them is to avoid them at the beginning. That doesn’t mean you need premature optimization; just keeping your code clean should be enough.

Optimizing Memory Usage and Preventing Memory Leaks

The Ruby Garbage Collector (GC) module serves as an interface to Ruby's mark and sweep garbage collection mechanism. Runtime garbage collection significantly influences memory management and recycling.

Although the GC operates automatically in the background as needed, the GC module allows you to manually invoke the GC when necessary and gain insights into the progress of garbage collection cycles. This module offers adjustable parameters that can help moderate performance.

Several frequently used methods within this module include:

start/garbage_collect: This method initiates a manual garbage collection cycle.
enable/disable: These methods activate or deactivate automatic garbage collection cycles. They return a boolean value reflecting the success of the operation.
stat: This method supplies a list of keys and values that describe GC module performance. We will delve into these metrics in detail in the following section.

To manually invoke the Garbage Collector, execute GC.<method> in your IRB or Rails console. For a deeper understanding of GC tuning, take a look at this article.

Alternative solutions for solving memory leaks

The Memory Profiler gem is an effective way of tracking the memory usage of ruby code.

If you have memory leaks despite handling memory correctly, the memory allocator methods may be the culprit. Ruby typically utilizes malloc calls for memory allocation, freeing, and reallocation, as mentioned earlier. An alternative implementation of these methods is jemalloc, developed by Jason Evans. Jemalloc proves to be more efficient in memory management compared to other allocators, mainly due to its focus on fragmentation avoidance, heap profiling, and comprehensive monitoring and tuning hooks.

Slow queries

In addition to the techniques that were already mentioned earlier in this article, there are other optimizations that can be done to solve slow queries.

Batch Processing: If you have a large dataset, consider breaking your queries into smaller batches to reduce the load on the database.

Use Database Transactions: Wrap multiple database operations in a transaction to ensure atomicity and reduce the number of commits.

Database Connection Pooling: Use connection pooling to efficiently manage database connections and reduce the overhead of establishing connections for each query.

Monitor and Profile: Regularly monitor your application's database performance and profile your queries to identify slow or inefficient queries. Use tools like Rails' built-in profiler (ruby-prof) or database-specific profiling tools.

Upgrade Database Versions: If you are using a relational database, ensure you are running the latest version with performance improvements and optimizations.

Remember that the effectiveness of these techniques may vary depending on the specific database system you are using (e.g., PostgreSQL, MySQL, SQLite, etc.) and the nature of your data and queries. Regularly test and benchmark your application's performance to ensure you're making the most significant improvements to your queries.

API payloads

If your application relies on external services or APIs, slow responses from those services can impact your application's performance.

Handling large amounts of data can slow down the application. Make sure to use pagination or limit the data retrieval to what is necessary.

How to make API payloads faster

Use Compression: Compress the payload using techniques like gzip or Brotli. This reduces the data size and speeds up transmission.

Limit Response Size: Only include essential data in the response. Avoid sending unnecessary fields to reduce the payload size.

Use Pagination: Implement pagination for large datasets. This way, clients can request only a subset of the data they need.

Minimize Nested Objects: Try to flatten the JSON structure by avoiding deeply nested objects. This reduces parsing overhead on the client side.

Enable HTTP/2: If possible, use HTTP/2, which allows multiple concurrent requests over a single connection, reducing latency.

Load Balancing: Implement load balancing to distribute API requests across multiple servers, preventing bottlenecks and ensuring optimal performance.

Monitor and Optimize: Continuously monitor your API's performance and usage patterns. Optimize and fine-tune as needed to accommodate changes in traffic.

By following these strategies, you can significantly improve the speed and efficiency of your API payloads, resulting in a better user experience and reduced server load.

Using AppSignal to track issues

AppSignal is a performance-monitoring and error-tracking tool. It helps developers identify and solve performance issues, track errors, and gain insights into the health and performance of their Ruby applications.

To use AppSignal to monitor and solve performance issues in a Ruby application, follow these steps:

Sign Up and Install AppSignal: First, sign up for an AppSignal account. Install the AppSignal gem in your Ruby application by adding it to your Gemfile and running bundle install. For more information, check the documentation.

Configure AppSignal: Set up the configuration for AppSignal in your application. This usually involves providing your AppSignal API key and specifying any custom settings.

Instrumentation: AppSignal automatically instruments common Ruby frameworks and libraries to collect performance data. Additionally, you can add custom instrumentation to specific parts of your code to gain deeper insights into their performance.

Monitoring and Error Tracking: With AppSignal installed and configured, it will start monitoring your application in real time. It will collect performance data, such as response times, database queries, and HTTP calls, and track errors and exceptions.

Web Dashboard: Use the AppSignal web dashboard to visualize your application's performance metrics and error occurrences. The dashboard provides an overview of your application's health and allows you to dig into specific issues.

Alerts and Notifications: Configure alerts and notifications in AppSignal to be informed of critical performance issues or errors immediately. AppSignal can send notifications via email, Slack, or other communication channels.

Performance Profiling: AppSignal also offers performance profiling, allowing you to analyze the performance of individual requests and find bottlenecks in your code.

Integrations: AppSignal integrates with popular tools like GitHub, Slack, and PagerDuty, making it easier to collaborate with your team and respond to issues promptly.

Resolve Issues: Use the data and insights provided by AppSignal to identify and resolve performance issues in your Ruby application. Optimize slow database queries, improve memory usage, and address other bottlenecks. You can see more here.

Continuous Monitoring: Keep AppSignal running in your production environment to continuously monitor your application's performance and detect issues as they arise.

By leveraging AppSignal's monitoring capabilities, you can gain a deeper understanding of your Ruby application's performance, identify potential problem areas, and proactively resolve performance issues before they impact your users.

Summary

People say ruby is slow because it is slow based on measured comparisons to other languages. Bear in mind, though, that "slow" is relative. Often, Ruby and other "slow" languages are fast enough.

But with the tuning performance advice, you can learn how to turn Ruby really fast, even compared to other compiled languages.

JIT Compilers for Ruby and Rails: An Overview

Renata Marques — Wed, 14 Sep 2022 14:29:24 +0000

A program is compiled at runtime using a different method from pre-execution compilation. This process is known as just-in-time compilation or dynamic translation.

In this post, we'll look at why JIT compilation can be a good choice for your Ruby on Rails app, before looking at some of the options available (YJIT, MJIT, and TenderJIT) and how to install them.

But first: how does JIT compilation work?

How a JIT Compiler Works

Just-in-time compilation is a method of running computer code that requires compilation while running a program.

This could entail translating the source code, but it's most frequently done by converting the bytecode to machine code, which is then run directly.

The code being executed is often continuously analyzed by a system using a JIT compiler. This identifies sections of code where the benefit of compilation or recompilation (in terms of speed) outweighs the cost.

Benefits of JIT Compilation for Ruby

JIT compilation combines some of the benefits (and shortcomings) of the two conventional methods for converting programs into machine code: interpretation and ahead-of-time compilation (AOT).

Roughly speaking, it combines the flexibility of interpretation with the speed of generated code, and the additional overhead of compiling and linking (not just interpreting).

JIT compilation is a type of dynamic compilation that enables adaptive optimization techniques, including dynamic recompilation and speed-ups tailored to certain microarchitectures. Due to a runtime system's ability to handle late-bound data types and impose security guarantees, dynamic programming languages like Ruby are particularly well-suited for interpretation and JIT compilation.

An optimizing compiler like GCC can more efficiently optimize instructions — a significant advantage of adopting a register-oriented architecture. Compilers operate on intermediate representation with register-based architecture.

Once your instructions reach an intermediate representation during compilation, GCC does additional passes to speed up the CPU's execution of your instructions.

JIT Compilers for Ruby: YJIT, MJIT, and TenderJIT

Now let's explore the different JIT compilers available for Ruby — YJIT, MJIT, and TenderJIT — and how you can set them up.

MJIT (Method-based Just-in-time Compiler) for Ruby

Vladimir Makarov implemented MJIT, and it was the first compiler methodology implemented in Ruby based on the C language. It works with Ruby 2.6, uses YARV instructions, and compiles instructions often used in binary code.

For programs that are not input/output-bound, MJIT enhances performance.

YJIT is better than this original C-based compiler in terms of performance. Ruby 3's JIT is the quickest JIT that MRI has ever had, made possible by the excellent work of MJIT.

How to Use MJIT

To use MJIT, you can enable the JIT in Ruby 2.6 and with the --jit option.

ruby --jit app.rb

If you skip this part, MJIT will show an error.

clang: error: cannot specify -o when generating multiple output files
MJIT warning: Making precompiled header failed on compilation. Stopping MJIT worker...
MJIT warning: failed to remove "/var/folders/3d/fk_588wd4g12syc56pjqybjc0000gn/T//_ruby_mjit_hp25992u0.h.gch": No such file or directory
Successful MJIT finish

A collection of JIT-specific settings included in Ruby 2.6 helps us understand how it functions. Run ruby --help to view these options.

In short, MJIT executes in a different thread and is asynchronous. It will begin just-in-time compilation following the first five runs of a calculation.

YJIT for Ruby on Rails

A recent JIT compiler called YJIT was released with Ruby 3.1. It promises a lot of improvements and better performance. Still a work-in-progress project designed by Shopify with experimental results, it must be used with caution, especially on larger applications.

With that in mind, YJIT enhances the performance of Ruby on Rails applications. The majority of real-world software benefits from the fast warm-up and performance enhancements provided by the YJIT basic block versioning JIT compiler.

A JIT compiler will be gradually built into CRuby as part of the YJIT project, eventually replacing the interpreter for most of the code execution.

Official benchmarks — see 'YJIT: Building a New JIT Compiler for CRuby' — show that YJIT improved performance over the default CRuby interpreter by:

20% on railsbench
39% on liquid template rendering
37% on activerecord

However:

Only about 79% of instructions in railsbench are executed by YJIT, and the rest run in the default interpreter.

Source: YJIT: Building a New JIT Compiler for CRuby

This means that a lot still needs to be done to improve YJIT's current results.

Even so, YJIT performs at least as well as the interpreter on every benchmark, even on the hardest ones, and reaches near-peak performance after just one iteration of every benchmark.

How to Use YJIT

Note: YJIT is currently limited to macOS and Linux on x86-64 platforms. Also, as mentioned, YJIT is not recommended for large applications (yet).

YJIT is disabled by default. If you want to enable it, first specify the --yjit command-line option.

You need to check if it is installed, so run ruby --enable-yjit -v. If warning: unknown argument for --enable:yjit'` shows up, you have to install it.

Then open irb and set RUBY_YJIT_ENABLE=1. You can exit and now, you're ready to use YJIT. The command ruby --enable-yjit -v must return something like ruby 3.1.0p0 (2021-12-25 revision fb4df44d16) [arm64-darwin21].

TenderJIT

With a design largely based off YJIT, TenderJIT is an experimental JIT compiler for Ruby. What's different about TenderJIT is that it's written in pure Ruby.

This is a demo project and the aim is to ship it as a gem. In the meantime, you can experiment with it, but bear in mind it's still a work in progress. Ruby 3.0.2 or later is required for TenderJIT.

How to Use TenderJIT

TenderJIT does not currently do method compilation automatically. To compile a method, you must manually configure TenderJIT.

Clone the repository and run the following commands:

$ bundle install
$ bundle exec rake test

You must set it manually on your code:

require "tenderjit"

def your_method
...
end

jit = TenderJIT.new
jit.compile(method(:your_method))

Each YARV instruction in the target method is read by TenderJIT, which then transforms it into machine code.

For more examples with TenderJIT, check one of these videos:
A JIT compiler for Ruby with Aaron Patterson and Hacking on TenderJIT!

Wrapping Up

In this post, we've taken a quick look at three JIT compilers for Ruby — MJIT, YJIT, and TenderJIT — and how to set them up. Each of the options is experimental and comes with its own limitations.

However, YJIT is the most mature at the moment, and it has the biggest potential to grow and scale. It demonstrates better performance over the other Ruby JITs, was developed with Ruby 3.1.0, and is quickly becoming an important part of CRuby.

Check out this post if you want to build your own compiler for Ruby.

Happy coding!

P.S. If you'd like to read Ruby Magic posts as soon as they get off the press, subscribe to our Ruby Magic newsletter and never miss a single post!

Mastering Ruby: Understanding Blocks, Procs, and Lambdas

Renata Marques — Mon, 13 Jul 2020 01:34:27 +0000

First, let's do a little introduction.

Ruby blocks are anonymous functions that can be passed into methods, a way of grouping statements, are passed to methods that yield them within the do and end keywords, and they can have multiple arguments.

A Proc object is an encapsulation of a block of code, which can be stored in a local variable, passed to a method or another Proc, and can be called.

Lambdas are anonymous functions, objects of the class Proc, they are useful in most of the situations where you would use a proc.

So, procs are blocks, and lambdas are procs, that's because all of them are closures, and they have similar characteristics thanks for this. What is a closure in Ruby? A closure is basically a function that:

Can be treated as a variable, i.e. assigned to another variable, passed as a method argument, etc.
Remembers the values of all the variables that were in scope when the function was defined and is able to access these variables even if it is executed in a different scope.

However, what really sets them apart?

What is the difference between Block, Proc, and Lambda?

First, Proc and lambda are objects but blocks are not.

Second, a lambda checks the number of arguments passed to it, while a proc does not.
This means that lambda will throw an error if you pass it the wrong number of arguments, whereas a proc will ignore unexpected arguments and assign nil to any that are missing.

Block uses the keyword yield, yield is a Ruby keyword that calls a block when you use it, the code inside the block will run and do its work.

Like in the example below, they all are Ruby blocks:

def print_message
  yield
end
print_message { puts "Block message" }

[1,2,3,4].each { |i| puts i }

[1,2,3,4].each do |i|
  puts i
end

Third,
when a lambda returns, it passes control back to the calling method,
when a proc returns, it does immediately, without going back to the calling method.

Like in the example below:

def best_movie_proc
  movie = Proc.new { return "Harry Potter!" }
  movie.call
  "Lord of the rings!"
end

puts best_movie_proc # prints "Harry Potter!"

def best_movie_lambda
  movie = lambda { return "Harry Potter!" }
  movie.call
  "Lord of the rings!"
end

puts best_movie_lambda # prints "Lord of the rings!"

See how the proc says "Harry Potter!", this is because it returns immediately, without going back to the best_movie_proc method.

Our lambda, however, goes back into the method after being called, so the method returns the last code it evaluates: "Lord of the rings!"

Calls

Block call

def method
  if block_given?
    yield
  end
end

method { puts "Block message" }

In the given example, we have a block passed with yield keyword.

Proc call

def method(proc)
  proc.call # or proc.()
end

my_proc = proc { puts "Proc message" }

method(my_proc)

In procs, we have two different syntaxes, with call and .()

Lambda call

def method(lambda)
  lambda.call # or lambda.()
end

my_lambda = -> { puts "Lambda message" }

method(my_lambda)

Because lambdas are procs, they also have the same call syntax, but they are different to create.

Conclusion

I hope it has become clearer for you to harness the power of Ruby closures.

Design Patterns with Ruby on Rails part 2: Query Object

Renata Marques — Mon, 22 Jun 2020 01:04:44 +0000

This post is the second part of a series of posts about design patterns with Ruby on Rails.
See other parts here:
Part 1

Query Object

The query object is a pattern that helps in decomposing your fat ActiveRecord models and keeping your code slim and readable by extracting complex SQL queries or scopes into the separated classes that are easy to reuse and test.

Naming Convention

Usually located in app/queries directory, and of course, can be organized into multiple namespaces.
Typical file name has _query suffix, class name has Query suffix.
e.g.:
Posts::ActivePostsQuery

module Posts
  class ActivePostsQuery
    def self.call
      Post.where(status: 'active', deleted_at: nil)
    end
  end
end

Class methods are more practical and easier to stub.

There is a way to divide this query into pieces and become even more reusable? Yes, chaining methods.

class PostsQuery
  def initialize(posts = Post.all)
    @posts = posts
  end

  def active
    @posts.where(active: true, pending: false)
  end

  def pending
    @posts.where(pending: true, active: false)
  end

  def deleted
    @posts.with_deleted
  end
end

This way:

query = PostsQuery.new
query.deleted.pending

Query objects and model scopes

I have the following scope defined in Post class:

class Post < ActiveRecord::Base
  scope :active, -> {
    where(section: ['web', 'mobile'], status: 'active', deleted_at: nil)
  }
end

and you want to extract this query and keep this behavior with Post.active, how can you do this?

The solution may seen obvious:

class Post < ActiveRecord::Base
  def self.active
    ActivePostsQuery.call
  end
end

but it adds more code and it is no longer visible in the scope definition.

Using a scope with query objects

Don't forget that your query object has to return a relation.

class Post < ActiveRecord::Base
  scope :active, ActivePostsQuery
end

Let's design our query object class:

class ActivePostsQuery
  class << self
    delegate :call, to: :new
  end

  def initialize(relation = Post)
    @relation = relation
  end

  def call
    @relation.where(status: 'active', deleted_at: nil)
  end
end

Now the query still available via Post.active scope.

Refactoring

Let's take a look at how to use query objects in practice when refactoring larger queries.

class PostsController < ApplicationController
  def index
    @posts = Post.where(active: true, deleted_at: nil)
      .joins(:authors).where(emails: { active: true })
  end
end

When is time to write tests for this action it won't be possible without consulting the database.
It gets easier after extracting this query to a separated class:

class ActivePostsWithAuthorQuery
  attr_reader :relation

  def self.call(relation = Post)
    new(relation).call
  end

  def new(relation = Post)
    @relation = relation
  end

  def call
    active.with_author
  end

  def active
    relation.where(active: true, deleted_at: nil)
  end

  def with_author
    relation.joins(:authors).where(emails: { active: true })
  end
end

and the controller can be now updated:

class PostsController < ApplicationController
  def index
    @posts = ActivePostsWithAuthorQuery.call
  end
end

Testing

The queries it's not a controller concern anymore, it simplifies your test, now you can simply call allow(ActivePostsWithAuthorQuery).to receive(:call).and_return(...) in your spec.

Conclusion

There is always a trade-off and moving to a separated class is not always a good idea, it adds a complexity layer to your code and has a maintainability cost, use this pattern with caution, and enjoy your extendable code.

Design Patterns with Ruby on Rails part 1: Introduction and Policy Object

Renata Marques — Mon, 22 Jun 2020 01:01:20 +0000

This post is the first part of a series of posts about design patterns with Ruby on Rails.
See other parts here:
Part 2

What are design patterns?

A pattern is the formalization of a problem/solution pair, used to make an object-oriented design decision.
In sum, patterns are a set of general rules to achieve something that is easily repeatable for other developers.

Why design patterns are important?

First, the purpose of a pattern is to codify existing design knowledge so that developers are not constantly reinventing the wheel.
Second, design patterns make communication between designers more efficient.
Untested fat models, controllers, helpers, and views are a technical disaster, if you are not testing your code, it’s harder to see the need for using patterns.
And how can you obtain design patterns benefits? applying these concepts:

Isolation: if the logic related to database queries is isolated then you can easily use stubs testing. The same rule applies to stuff like indexing or the 3rd part code.
Readability: you can say what the given code is doing in general only by reading the class name.
Extendability: it’s easy to modify the existing code and there is no need to change the logic in many places.
Single Responsibility: a method or a class has to be responsible only for one action.
Testability: thanks for the benefits mentioned, it becomes easier because we have to test only a small portion instead of large methods, connect to the external services and do the business logic at the same time.

Policy Object

Let’s begin with the simplest pattern, policy object is a pattern to deal with permissions, roles, and policies, you can use each time you have to check if something or someone is allowed to do the action. Gems like pundit, Cancan, and Cancancan implement this pattern.

Naming Convention

The filename usually has _policy suffix applied and the class with Policy at
the end. The method names always end with ? character.
e.g.: PostsPolicy#web_section?

A pure policy object is defined by these simple rules:

the return has to be a boolean value
the logic has to be simple
inside the method, we should only call methods on the passed objects

e.g.:

class PostsPolicy
  def initialize(post)
    @post = post
  end

  def web_section?
    active? && @post.section == ‘web’
  end

  def active?
    @posts.where(active: true, pending: false)
  end
end

Simply calling other methods and comparing using the data, this is the main purpose of policy object.
They are light, simple pure Ruby objects, used for managing permissions across the project.
They are also easy to test and a perfect replacement for complex conditions.

An example of a complex condition:

class PostsController < ApplicationController
  def create
    if @blog.mode == ‘live’ && @blog.authors.size > 0
    && (current_user.role == ‘admin’
    || (current_user.role == ‘moderator’ && current_user.verified_email))
      # create
    end
  end
end

The above condition checks are long, ugly, and unreadable, the policy object pattern can be applied.

Let’s begin creating PostsCreationPolicy

class PostsCreationPolicy
  attr_reader :user, :blog

  def initialize(user, blog)
    @user = user
    @blog = blog
  end

  def self.create?(user, blog)
    new(user, blog).create?
  end

  def create?
    blog_with_authors? && author_is_allowed?
  end

  private

  def blog_with_authors?
    blog.mode == ‘live’ && blog.authors.size > 0
  end

  def author_is_allowed?
    is_admin? || moderator_is_verified?
  end

  def is_admin?
    user.role == ‘admin’
  end

  def moderator_is_verified?
    user.role == ‘moderator` && user.verified_email
  end
end

our controller with the policy object looks like this:

class PostsController < ApplicationController
  def create
    if PostsCreationPolicy.create?(current_user, @blog)
      #create
    end
  end
end

Policies in models

Another way of refactoring complex queries is by creating small policy methods on the model class:

class User < ActiveRecord::Base
  def is_admin?
    role == ‘admin’
  end

  def is_moderator?
    role == ‘moderator’
  end

  def is_authorable?
    return true if is_admin?

    is_moderator? && verified_email
  end
end

class Blog < ActiveRecord::Base
  def live?
    mode == ‘live’
  end

  def any_authors?
    authors.any?
  end

  def publishable?
    live? && any_authors?
  end
end

now we can refactor our controller:

class PostsController < ApplicationController
  def create
    if @blog.publishable? && current_user.is_authorable?
     # create
    end
  end
end

As you can see, it’s a much smaller and readable way to write the same logic.

Conclusion

The policy pattern is a small concept that gives big results. Consider apply policy object each time you have to deal with complex conditions.
When it comes to testing with RSpec, you don’t need to use database records, your policies are pure-ruby objects and your tests are going to be fast.