DEV Community: Haseeb Annadamban

SQL injection in Rails - Learn from an attacker’s shoes

Haseeb Annadamban — Fri, 04 Aug 2023 19:13:16 +0000

Imagine a scenario where an attacker gains unauthorized access to your application's database, manipulates data, bypasses authentication measures, or even executes malicious code on the server. Sounds like a nightmare, right? This is the harsh reality of SQL injection, a security vulnerability that can wreak havoc on any application. SQL injection, or SQLi, typically occurs when unescaped user data is passed to an SQL query, opening the door for attackers to exploit the system. This guide will guide you through the intricacies of SQL injection in Rails applications, helping you understand the vulnerability itself and how to prevent such a disastrous scenario.

It is important to note that SQL injection is a preventable vulnerability, and developers should take care to ensure that their code is secure and free from vulnerabilities. Regular security audits and testing can also help to identify and address vulnerabilities before they are exploited by attackers.

Understanding SQL Injection

SQL injection can happen when a web application uses input fields to construct SQL queries without properly sanitizing the input. If an attacker can manipulate these queries, they can potentially view, modify, or delete data from the database. This is because we are constructing SQL queries by using another language. Let me explain with an example.

This is a classic login bypass example, which is easy to understand. The following query uses user supplied username and password to check if they match, so that the app can proceed to login the user.

User.where("username = '#{params[:username]}' AND password = '#{params[:password]}'")

If we give username as “admin” and password as “password”. It will generate the following SQL query.

SELECT * FROM users WHERE (username = 'admin' AND password = 'password');

But instead, If an attacker enters ' OR '1'='1'); -- as the username and leaves the password field blank, the SQL query becomes:

SELECT * FROM users WHERE (username = '' OR '1'='1'); -- AND password = '');

Since '1'='1' is always true, this query will include all usernames. And since -- is for used for commenting in SQL, the entire password part will get commented out, effectively bypassing the login mechanism.

Another way to do it is by passing the same ' OR '1'='1' as both username and password. It will work similarly.

I am not going to build suspense on how to fix it. It can be simply fixed by making sure the values are properly sanitized. It can be done in two ways:

Pass params as hash

User.where(username: params[:username], password: params[:password])

Use prepared query

User.where("email = ? AND password = ?", params[:email], params[:password])

This way, Rails will make sure your data is passed properly.

Is it that simple?

Yes, Rails makes it so easy to handle this. That's right!. But I have few more things to discuss.

Be careful of the injection vector

Injection vector is the specific location in the code where user-supplied input is improperly included or used in a command or query that is then executed or processed. Since we are using Rails, This is not only in where part of queries. In Rails it is possible in many other methods. Basically any method that accepts raw SQL as input is vulnerable.

Here are two examples. This is nowhere near a complete list. but serves as an example for different class exploitation.

Example 1: `group` method

If your query includes a group method and you pass a user submitted value to it. There could be this vulnerability. Lets consider this query,

User.where(subscribed: true).group(params[:group])

if we pass a proper parameter, such as name the generated SQL query will be similar to,

SELECT "users".* FROM "users" WHERE "users"."subscribed" = 1  GROUP BY "users"."email";

resulting in a proper group by statement in the query.

But, if an attacker sends the value of params[:group] as name UNION SELECT * FROM users, the generated query will become

SELECT "users".* FROM "users" WHERE "users"."subscribed" = 1  GROUP BY email UNION SELECT * FROM users;

As you can see the "users.name" part is no longer present and instead it generated an sql query to return all users.

Other methods similar to group

# .join
# .lock
# .select

Example 2. `calculate` method

You might be familiar with methods like sum, maximum etc. All these methods are shortcuts to calculate method.

User.where(admin: true).calculate(:sum, params[:column]) # is the same as User.sum(params[:column])

Suppose income is a valid column in the table. If a malicious user passes the value of params[:column] as income) from users where 'random_string' != ? --; they will be able to get the sum of income of all users.

Let me explain how:

-- is comment in sql. Here we are using sql injection to set all remaining parts of the SQL as invalid. and we included from users to address the commented out from part. See the generated SQL

SELECT SUM(income) from users where 'random_string' != ? --;) FROM "users" WHERE "users"."admin" = ?

2.The 'random_string' != ? is a technique to bypass the bind parameter error in Rails prepared queries by giving some random parameter to bind to, but making the result always true. Here it is used to bypass the internal bind parameter in .where(subscribed: true)

So, Be careful of the attack vectors. Any method that can accept raw SQL is really dangerous in Rails.

Misuse of the sanitize Method and Its Consequences

The sanitize method is good for preventing XSS, but it does not prevent SQL injection.

Let’s take the last example. But "protected" with sanitize method.

User.where(subscribed: true).sum(sanitize(params[:column]))

Sanitize method is for stripping HTML tags. So, it will try to strip any html tags. In this case, if the attacker passes income) from users where 'random_string' != ? --;, There are no HTML tags in it. So, it will simply return the string as it is. Hence the attacker is successful at SQL injection.

SQL injection can happen from stored data too

SQL injection happened from stored data is called a “Second order SQL Injection”. Stored or not, Any data from user should be handled carefully.

Let’s assume we have the following query to get payment details. Here, we allow users to provide payment details, which we will store in database. Here payment_org_secret is a unique uuid which will be present in all payments by an organization.


payment_data = Payment.where("payment_org_secret=#{external_payment.external_org_secret}")

Let’s assume if a user stored external_org_secret as ' OR '1'='1. It will return all payment information of all the organizations regardless of how secure the “secret” is.

Therefore, it's always good practice to use prepared queries or a hash-based format in all instances. We are not sure about where our methods will be used in future. especially when it is really easy to protect from this.

payment_data = Payment.where(payment_org_secret: external_payment.external_org_secret)

Passive detection of SQL injection

brakeman is a static code analyzer especially designed for detecting vulnerabilities in Ruby on Rails applications. It scans your codebase and identifies potential security vulnerabilities, including SQL injection attacks. Brakeman can be run as part of your continuous integration pipeline to ensure that your application remains secure throughout its development lifecycle. But aware that it is too proactive, it prefers to have false positives than to have security vulnerabilities in your app.

Additional level of defenses

Use strong parameters
Use a separate account with minimum privileges for your web app in the database.
Properly validate your data.

Conclusion

In conclusion, SQL injection is a serious security vulnerability that can lead to unauthorized access to sensitive data, bypassing of authentication measures, and execution of malicious code on the server. However, it's a preventable vulnerability that can be mitigated with careful coding practices and regular security audits.

When using Rails, it's crucial to avoid incorporating raw SQL into your queries. Methods such as where, group, join, lock, select, and calculate can all be potential injection vectors if not used properly. Even stored data can be a source of SQL injection, emphasizing the importance of treating all user data with caution.

While Rails provides tools to make handling these issues easier, it's ultimately up to developers to use these tools correctly. The sanitize method, for instance, is effective against XSS but not against SQL injection. Therefore, understanding the nuances of these tools and how to use them effectively is key.

Broken Access Control: What Is It and Why Does It Matter in your Rails application?

Haseeb Annadamban — Wed, 02 Aug 2023 14:28:46 +0000

Access control is the backbone of web application security, ensuring that users only have access to the resources they are authorized to use. When this control mechanism is poorly implemented or overlooked, it can lead to disastrous consequences, leaving your application exposed to unauthorized access, data breaches, and even complete compromise. Understanding what is broken access control is essential for your application security. How serious is this?, Let’s find out.

What is access control

Access control is all about setting some rules for the requesting user can access a particular resource or action. Sometimes with additional decisions like how much we can allow to access. It is about restricting or allowing certain actions or resources based on user permissions and roles.

What is broken access control

Broken access control refers to either of not having expected level of access control or when an attacker is able to bypass the intended access control restrictions and gain unauthorized access to sensitive resources or actions.

This can happen due to various factors such as

Insufficient authorization checks
Lack of input validation
Poorly configured roles or permissions

The Basics of access control

I will discuss about the basics of access control first.

Authentication

Authentication is the first step in this. This process involves verifying the identity of a user. In Rails, authentication is often implemented using gems like Devise or Sorcery. Implementation of authentication mechanisms like password based authentication is beyond the scope of this.

Once a user is authenticated, they receive a session token or a cookie that allows them to be recognized as a valid user for subsequent requests. The most important thing from security perspective is, we should verify if the current user is authenticated for all sensitive resources. You can follow two approaches when using devise gem.

1. Authenticate when it is needed
we will use before_action :authenticate_user! wherever it is needed.
we will use before_action :authenticate_user! wherever it is needed.

class SensitiveController < ApplicationController
  before_action :authenticate_user!
end

2. Authenticated by default

in this strategy we will define authenticate_user! on a base class,

class BaseAuthenticatedController < ApplicationController
  before_action :authenticate_user!
end

And then all controllers that extends BaseAuthenticatedController (name it as per your requirement) will be authenticated by default:

class SensitiveController < BaseAuthenticatedController
  # No need to add before_action :authenticate_user! here
end

For controllers with only public methods we will continue to extend ApplicationController:

class HomePageController < ApplicationController
end

And we will skip authentication for public methods where BaseAuthenticatedController is used:

class SensitiveController < BaseAuthenticatedController
  skip_before_action :authenticate_user!, only: [:public_action]

  def sensitive_action
    # Authenticated as usual
  end

  def public_action
    # We explicitly skipped authentication for this endpoint.
  end
end

Authorization

Authorization is the process of making sure the authenticated user has the necessary permissions to perform certain actions or access specific resources. In rails we have two equally famous gems for this at the time of writing this. CanCanCan and Pundit. I will mention either pundit or will write authorization code directly here for the sake of simplicity. Pundit gem focuses on providing explicit policy classes for each resource, making the authorization rules more organized and maintainable.

The basic guard against forgetting to add authorization in pundit is with verify_authorized. It will raise an error if authorization is not present for an action in the controller.

I would recommend to add it in BaseAuthenticatedController or ApplicationController to ensure we are always authorized.

class BaseAuthenticatedController < ApplicationController
  include Pundit::Authorization
  before_action :authenticate_user!
  after_action :verify_authorized
end

Access control implementation

There are several ways to implement proper access control. Role based access control is considered the base of it all. I will explain about it and will mention some others.

Role Base Access Control (RBAC)

RBAC is a widely used access control model that focuses on organizing and managing user permissions based on their roles within an organization or system. In RBAC, users are assigned specific roles, and each role is associated with a set of permissions or privileges that determine what actions the user is allowed to perform on various resources.For example, you might have roles like "admin," "user," and "guest." Admins would have more privileges than regular users, who, in turn, would have more privileges than guests.

To implement RBAC

You might add a role column as a string in your users table:

class AddRoleToUsers < ActiveRecord::Migration
  def change
    add_column :users, :role, :string, default: "user"
  end
end

In the policy, Check if a user has a specific role

class ArticlePolicy < ApplicationPolicy
  #
  # ....
  #
  def update?
    user.present? && (user.role == "admin" || user.role == "editor") # Only admins and editors can update articles
  end
end

Check this in your controller

class ArticlesController < BaseAuthenticatedController
  def update
    @article = Article.find(params[:id])
    authorize @article # This will make sure only the editors and admins are able to access the update
  end
end

Others

Attribute-Based Access Control (ABAC)
ABAC is a more flexible access control model that takes into account various attributes of the user, the resource, and the environment to make access decisions. It uses a set of policies based on attributes such as user attributes (e.g., age, location, department), resource attributes (e.g., sensitivity, type), and environmental attributes (e.g., time of day, location). ABAC allows for more fine-grained and dynamic access control compared to RBAC.

Mandatory Access Control (MAC)
MAC is a strict access control model typically used in high-security environments such as government and military systems. Access decisions are based on sensitivity labels attached to users and resources. MAC enforces a hierarchical system of clearances and classifications, and it requires a centralized authority to manage access decisions.

Discretionary Access Control (DAC)
DAC is a more permissive access control model where resource owners have the discretion to control access to their resources. In DAC, access decisions are based on the identity of users and the permissions granted by the resource owner. It is commonly used in file systems and some traditional network access control scenarios.

Rule-Based Access Control (RBAC)

Rule based access control is a variation of RBAC, where permissions are defined using rules or policies. The rules can be more complex than simple role-to-permission mappings and can take various attributes and conditions into account. Access decisions are made based on rules defined by administrators or security experts. These rules can be more expressive and granular than typical RBAC models, providing greater control over access rights.

Hierarchical Role-Based Access Control (HRBAC)
HRBAC is an extension of RBAC that introduces role hierarchies. Roles are organized in a hierarchical structure, and permissions can be inherited from higher-level roles to lower-level roles. HRBAC simplifies the management of permissions and allows for a more flexible access control scheme.

History-Based Access Control (HBAC)
HBAC is an access control model that takes into account the historical behavior of users to make access decisions. It uses past user behavior patterns and access logs to determine access rights. HBAC is suitable for detecting anomalies and preventing insider threats.

Now that we have some strong basics, let’s proceed to discuss the vulnerabilities that can happen due to broken access control.

Insecure Direct Object Access (IDOR)

IDOR can be described as an attacker being able to bypass authorization and directly accesses resources in the system because user supplied arguments are used to give access to a particular resource or action and the attacker replaced them to some other value.

One reason for this is Not having an authorization at all on the server. It won’t help even if you have client side validation. Client side is not the right place to have the authorization mechanism. Consider this controller's update action which updates an article

class ArticlesController < ApplicationController
  before_action :authenticate_user!

  def update
    @post = Post.find(params[:id])
    @post.update(post_params)
    redirect_to @post
  end
end

In this example, the update actions is vulnerable to IDOR. Although the authenticate_user! method ensures that a user is logged in, it doesn't verify that the logged-in user is the owner of the post they're trying to access.

This means that if an attacker can guess or otherwise discover the ID of a post that doesn't belong to them, they can simply add it in the URL path and edit any article.

PATCH /articles/:any_id

This can be fixed by adding a PostPolicy and using that

class PostPolicy < ApplicationPolicy
  def update?
    record.user == user
  end
end

class ArticlesController < ApplicationController
  before_action :authenticate_user!

  def update
    @post = Post.find(params[:id])
    authorize @post # Authorization here.
    @post.update(post_params)
    redirect_to @post
  end
end

Second reason for IDOR is Inadequate Access Control. If an application doesn't implement access controls properly, a user might be able to escalate their privileges and access resources they're not supposed to. Usually happens due to bugs in code or developers not understanding or forgetting complex rules properly.

For example, Let's assume a forum software where A user can update if user is owner of the post or have more than 1000 reputation or is assigned an editor for a tag to edit a post in a forum but we have a bug in our code which instead of checking if user is editor for a tag, it checks if the user is added as an editor for any tags

class PostPolicy < ApplicationPolicy
  def update?
    user_is_owner? || user_has_high_reputation? || user_is_tag_editor?
  end

  private

  def user_is_owner?
    record.user == user
  end

  def user_has_high_reputation?
    user.reputation > 1000
  end

  def user_is_tag_editor?
    # Here it checks if the user is an editor for any tag. It is a bug
    user.editor_tags.any?
  end
end

Having access control related tests are a great way to avoid this type of errors.

Privilege Escalation

Attackers could exploit weak access controls to elevate their privileges, gaining more access and control over a system than they should have. There are two types of privilege escalation.

Horizontal privilege escalation

This occurs when a user gains unauthorized access to another user's account of the same privilege level, often leading to unauthorized actions within the system.

For example in this comments controller we verify only admin users are able to access a post that is not a comment.

class PostsController < BaseAuthenticatedController
  before_action :set_post
  before_action :authorize_post_access

  def update
    if @post.update(post_params)
      redirect_to post_path(@post)
    else
      render :edit
    end
  end

  private

   def set_post
     @post = Post.find(params[:id])
   end

   def authorize_post_access
    if !current_user.admin? && @post.post_type != "comment"
      redirect_to root_path, alert: "You do not have access to this resource."
    end
   end
end

Even though it is not possible to update other type of posts other than comment when a user is a non admin user, they can still update other users comments simply changing the comment id.

One way to fix is by adding an additional check in authorize_post_access

def authorize_post_access
   return if current_user.admin?

   if @post.post_type != "comment" || @post.author != current_user
     redirect_to root_path, alert: "You do not have access to this resource."
   end
end

Vertical privilege escalation

This occurs when attackers exploit weak access controls to gain access to higher privilege levels, such as administrative accounts, allowing them to control the entire system.

In this example of a website like twitter, we have two endpoints block_user where any user can block any other user and ban_user where admin users can ban other users.

class UserActionsController < ApplicationController
  before_action :authenticate_user!

  # ...

  def block_user
    @user = User.find(params[:id])
    if current_user.id != @user.id  # Prevent blocking yourself
      @user.update(user_actions_params)
      redirect_to @user, notice: "User has been blocked."
    else
      redirect_to @user, alert: "You cannot block yourself."
    end
  end

  def ban_user
    @user = User.find(params[:id])
    if current_user.admin?
      @user.update(user_actions_params)
      redirect_to @user, notice: "User has been banned."
    else
      redirect_to @user, alert: "You do not have permission to ban users."
    end
  end

  private

  def user_actions_params
    params.require(:user).permit(:blocked, :banned, :follow)  # Mistake: used same method for both actions
  end
end

Since update_user_params is used inside both ban_user and block_user , an attacker can use the param banned=true to ban any users.

Another example will be having a bug using which a user can make themselves admin. Suppose we have an option to update the user profile and the user is able to update the role parameter from there. Then an attacker can change the role to become an admin.

Conclusion

In conclusion, understanding and addressing broken access control is paramount for the security and integrity of your Rails application. This often overlooked vulnerability can lead to disastrous consequences, including unauthorized access, data breaches, and compromised user privacy. By implementing robust access control mechanisms and following best practices, you can significantly mitigate the risk of such attacks.

Prioritize user authentication, authorization, and having an access control mechanism suitable for your app to ensure that only authorized users can access and modify sensitive resources.

Notes on Performance Optimization in Rails Applications

Haseeb Annadamban — Wed, 26 Jul 2023 20:25:10 +0000

Building web applications in Rails is a rewarding journey. However, as your audience grows and your platform gains traction, you might have noticed a new challenge on the horizon: the need for performance optimization. With more users accessing your application, ensuring its responsiveness and scalability becomes more important to maintain a seamless user experience.

In this post, we'll delve into the world of Performance Optimization in Rails Applications, exploring the best practices, tools, and techniques to make your application fast and capable of handling increased user traffic with ease.

I work as a Rails performance consultant, Helping businesses optimize their Rails applications. I fix performance issues from simple N+1 issues to complex memory issues. If you're looking to improve your app's performance, the strategies discussed in this post can be a great starting point. Performance optimization isn't just about faster load times and smoother interactions; it's about enhancing user satisfaction, boosting search engine rankings, and, ultimately, solidifying your position in an ever-competitive digital landscape. This blog post aims to guide Rails developers through performance optimization strategies, where performance is much more than just speed - it includes scalability, reliability, and efficiency. This post discusses how to avoid having these issues by following best practices and early-stage optimizations. I will avoid anything that is treated as over-engineering or premature optimizations. Instead, These are treated as best practices in the Rails community.

In my experience, Performance optimizations in Rails are usually a mix of three factors

Database optimization
Memory Optimization
Use of the correct algorithm

Database optimization

The database is often the primary bottleneck, causing lags in your Rails application's performance. Optimizing your database can greatly improve the speed and efficiency of your Rails app. These are some of the common issues to consider

N+1 Query problem

The issue

The N+1 query problem is a common performance issue that can significantly slow down Rails applications. This problem occurs when you're pulling in associated data for each record individually, instead of retrieving all at once. For instance, when you're fetching a list of posts and their comments, Rails might make one database call for the posts and then another for each post's comments.

The solution

Use eager loading techniques like includes, preload, and eager_load. These methods allow you to load all the associated records in one go, thus reducing the number of queries.

Let's consider an example where we have a User model and a Post model. Each User has many Posts.

Without eager loading, if we were to get all posts for all users, we might write something like:

users = User.all

users.each do |user|
  puts "User: #{user.name}"
  user.posts.each do |post|
    puts "Title: #{post.title}"
  end
end

This code has the N+1 query problem. The User.all query pulls all users, and then for each user, we're making another query to fetch their posts with user.posts. If we have 100 users, this results in 101 queries - one for fetching users and 100 for fetching the posts of each user.

To solve this N+1 query problem, we can use eager loading with the includes method:

users = User.includes(:posts)

users.each do |user|
  puts "User: #{user.name}"
  user.posts.each do |post|
    puts "Title: #{post.title}"
  end
end

Now, only two queries are executed, regardless of the number of users. One query fetches all users and another fetches all posts associated with those users. This is a significant improvement, especially as the number of users increases.

Database indexing

Database indexing is a powerful tool in a developer's arsenal that can greatly boost application performance when implemented correctly.

Think of a database as a massive library and the data as its collection of books. Without any system of organization, finding a specific book would involve going through the entire collection, which is not efficient. This is where an 'index' comes into play. In the context of a library, an index could be a catalogue system which allows you to locate a book by its title, author, or subject etc. This same concept applies to databases.

In a database, an index allows the database server to find and retrieve specific rows much faster through the index. However, creating an index requires additional disk space, and maintaining an index can slow down data-writing operations. This is why it's crucial to use indexes carefully and only on columns that will be searched or sorted frequently or if you feel indexing will boost performance significantly.

Ruby on Rails provides support for database indexing out of the box through ActiveRecord migrations. When you create a new migration to add an index, Rails will generate the SQL necessary to add the new index to your database.

To add an index, you can use the add_index method in your migration:

add_index :table_name, :column_name

Indexes are not only for individual columns. Rails also supports composite (multi-column) indexes. These can be useful for queries that filter or sort by multiple columns.

Effective indexing requires a solid understanding of the data and the queries your application makes. An index might greatly speed up data retrieval for one type of query but might be useless or even detrimental for another.

Knowing the distribution and characteristics of the data in your table is essential. If a column has a high degree of uniqueness, it's usually a good candidate for indexing. Conversely, if a column has very low uniqueness, an index on this column may not be beneficial.

While indexes can speed up data retrieval, they come at a cost. Every time a write operation (INSERT, UPDATE, DELETE) occurs on a table, all indexes on that table must also be updated. Therefore, while adding more indexes can speed up read operations, it can also slow down write operations. You need to strike a balance based on your application's specific read-write patterns.

We have an option to explain our queries in rails. See this guide for more information. You can use it to effectively determine if indexes are needed whether it's using your indexes effectively.

Common query optimizations

`select` and `pluck`

When you only need specific fields, you can use the select and pluck methods to increase performance.

The select method limits the retrieved data to specified columns:

User.select(:name, :email)

The pluck method retrieves specified columns a converts them into and converts them into array of values, bypassing the creation of ActiveRecord objects. It should be at the end of your query because no further chaining is possible since it returns an array:

User.pluck(:name, :email)

Batch processing

ActiveRecord’s all, find, and where methods load all matching records into memory at once. While this is fine for small datasets, it could lead to high memory usage when dealing with large tables. The find_each and find_in_batches methods offer a way out by retrieving records in batches. Both the methods below fetch a 1000 records only at a time

User.find_each(batch_size: 1000) do |user|
  # Process individual user
  puts user.name
end

User.find_in_batches(batch_size: 1000) do |group|
  # Process a batch of users
  group.each do |user|
    puts user
  end
end

You can avoid callbacks if you know what you are doing

ActiveRecord callbacks can cause a significant performance hit, especially when processing large batches of records. If you're performing a mass update/insert and the model has complex callbacks, consider bypassing them using the methods like update_all or insert_all. However, while avoiding callbacks can improve performance, particularly for bulk operations, it's crucial to understand when doing so could be harmful to your application. Here are some scenarios when avoiding callbacks might be a bad idea. It can skip business logic added as callbacks and data validations and important side effects added through callbacks like before_update etc.

Try to minimize db access

ActiveRecord is powerful and flexible. However, it's easy to write inefficient queries without realizing it. So, when writing queries keep an intention to reduce the number of db hits

# Don't do this
# It will hit db for each record
Post.all.select{ |p| p.published? }.each { |p| puts p.title }

# This hits db only once
Post.where(published: true).each { |p| puts p.title }

Counter cache

When you frequently need to get the count of associated records, consider using counter_cache, which stores the count in a column on the parent record.

# In your migration
add_column :posts, :comments_count, :integer, default: 0

# In your model
class Comment < ActiveRecord::Base
  belongs_to :post, counter_cache: true
end

This eliminates the need for a count query each time you need the number of associated records, saving memory and processing time.

Caching

Proper use of caching can drastically improve the speed and responsiveness of your application. There are multiple types of caching in rails. Rails has an awesome guide about caching

Pagination

The basic idea behind pagination is to only load a subset of data at a time, rather than trying to load all records into memory at once, which can have a significant impact on performance.

Rails provides some built-in methods for easy pagination with the use of the limit and offset commands

# This will retrieve the first 20 records
@posts = Post.limit(20)

# This will retrieve the next 20 records
@posts = Post.limit(20).offset(20)

There are also gems like kaminari and will_paginate that provide more powerful and customizable pagination features, including generating pagination links in views and handling edge cases.

Advanced: Read replicas

When your application scales, your single database server might become a bottleneck due to the increasing number of read operations. In this case, Use read replicas to distribute the load. A read replica is a copy of the primary database that can handle read traffic. This can greatly increase the performance of your Rails application. You can easily configure ActiveRecord to direct read queries to the replica.

production:
  primary:
    adapter: postgresql
    database: my_primary_database
  replica:
    adapter: postgresql
    database: my_replica_database
    replica: true

There are even more advanced strategies like Sharding, Denormalization etc. They are outside the scope of this post.

Memory optimization

Many of the techniques discussed earlier like Avoiding N+1 queries, use of 'pluck' and 'select' methods, Batch processing etc helps to optimize memory usage. Here are some other strategies

Use latest stable version of ruby and rails

First and foremost, ensure you're using the most recent version of Ruby. Ruby's garbage collector, responsible for freeing up memory that is no longer in use, has been significantly improved in recent versions. The latest Ruby versions have a more efficient garbage collector that helps reduce memory usage and enhance performance.

Utilize lazy loading when working with large collections

collections are eager-loaded by default in ruby, meaning all objects in the collection are loaded into memory when the collection is initialized. When working with large collections, this can lead to substantial memory consumption.

# The program will get stuck when you run this. Tip: Hit ctrl+c
(1..Float::INFINITY).map{ |n| n * 2 }.first(50)

# This will return data correctly
(1..Float::INFINITY).lazy.map{ |n| n * 2 }.first(50)

Use symbols instead of strings where appropriate

In Ruby, symbols are immutable and unique, which makes them memory efficient. When the same symbol is used multiple times, Ruby points to the same object in memory, whereas with strings, a new object is created every time, consuming more memory.

Limit session data

Storing excessive data in sessions can consume significant server-side memory, especially if you're using a server-side session store. Try to limit session data to only what's necessary and avoid storing large objects or data sets.

Use background jobs for long running processes

Long running processes can significantly consume memory and CPU. If the task does not need to be completed immediately, consider moving it to a background job. Background jobs can be a great way to move resource-intensive tasks, such as sending emails or processing images, out of the request-response cycle. This ensures your users aren't left waiting for these tasks to complete and your application remains responsive.

Rails has built-in support for creating background jobs using Active Job. Which internally uses tools like Sidekiq etc. It will be better to use Sidekiq directly for advanced uses,

Use of correct algorithm

Proper algorithm selection and implementation can have profound effects on the performance of your application. Remember that a well-optimized application not only provides a better user experience but also consumes fewer resources, which leads to cost-effectiveness in the long run.

Use built-in ruby methods

When it comes to improving performance, one of the most important steps you can take is to use built-in Ruby methods wherever possible. Ruby has an extensive library of built-in methods that are optimized for performance and memory usage.

Built-in methods are already compiled and optimized, making them faster than custom methods. They also tend to be more reliable and less prone to errors, as they've been extensively tested and refined over the years.

A few examples of built-in Ruby methods include sort, map, select, and reject. These methods can be used for a wide range of tasks, including sorting arrays, transforming data, and filtering data.

Memoization in Rails

Memoization is a powerful technique that involves storing the results of expensive function calls and reusing the cached result when the same inputs occur again. This technique can significantly improve the performance of your Rails application by avoiding redundant calculations or database queries.

In Rails, you can achieve memoization using the ||= operator.

def expensive_method
  @result ||= ExpensiveOperation.new.call
end

Avoiding nested loops

Nested loops can be a major source of performance issues in your Rails application. Each additional level of nesting increases the complexity of the algorithm, leading to exponential growth in execution time as the size of your data sets increases.

While there are situations where nested loops are necessary, they should be avoided whenever possible. If you find yourself writing nested loops, it may be worth looking for a different approach or using a more efficient data structure.

For example, you can often replace nested loops with Ruby's built-in methods or use a hash table to look up values in constant time, rather than looping through an array.

How to find what's the issue with a slow endpoint

If you you are truly having this issue and want to learn more about it fixing performance issues in Rails apps, Then it is actually a long topic that won't fit in one or two blog posts. I have a book recommendation: The Complete Guide to Rails Performance by Nate Berkopec.

When testing locally, Use rack-mini-profiler. You will be able to use it to find issues like N+1 and slow queries easily. It has other functionalities like memory profiling, flamegraph etc. It can be run in production too if needed. Also you can run it locally making rails run in production mode.

You can make use of monitoring tools (APM) like newrelic in production to continuously monitor your performance metrics.

Improving performance of specific slow SQL query.

Sometimes you get alerts about a specific database query getting timing out. In most of these cases for web apps indexing will improve performance significantly. You will have to check where to add index using #expalin method. I have detailed two part blog series on this topic. You can start here: https://haseebeqx.com/posts/rails-explain-analyze-explained/

Conclusion

Rails performance optimization is a vital aspect of developing efficient and scalable applications. It's a continuous process that requires periodic review and adjustment. Remember, it's not about creating an application that merely functions, but one that flourishes under load, provides a seamless user experience, and stands the test of time.

How to run a ruby web server on your android device (without root).

Haseeb Annadamban — Mon, 20 Jun 2022 13:48:20 +0000

Here is how you can quickly run a ruby based web server in your android device. There are many development activities you can do in your android device. This is an easy one. So, I will be quick without going into details. I expect you know basics of linux terminal.

Step 1 : Termux

Termux is a linux terminal for your android. It also includes a package manager using which you can install many packages including ruby. The first step is to download and install termux. Do not download from play store. The play store version is pretty outdated. Get it from FDroid or github

Step 2 : Packages

pkg is the package manager available in your Termux. We need to install ruby and openssl
open termux and type

pkg install ruby openssl

It will install ruby and all required packages like clang.

Step 3 : Editor

I prefer vi editor. If you are new to vi editor, Please make sure you know the basics and especially how to exit vim 😀

pkg install vim

Step 4 : Gems

First we will install thin web server

gem install thin

Then we will install sinatra framework. Sinatra will help us to quickly create a hello world example website.

gem install sinatra

Step 5 the server.rb

First create a directory for convenience.

mkdir ruby-example

Then go there,

cd ruby-example

Now create server.rb with the following content

vi server.rb

require 'sinatra'

get '/' do
  'hello '*300
end

Now exit vim using the esc key provided in the UI and then :wq - Save and exit.

Step 6 : Visit the website

in your chrome web browser go to http://localhost:4567
It. You will see 300 hellos in your browser.

Step 7 Stopping the server

Press the CTRL button in the UI and then c to exit

Now what

This is not limited to ruby. Using termux you can do a lot more developmental activities in your android device. This is very powerful.

How to implement Authorization in elixir and phoenix

Haseeb Annadamban — Sun, 06 Oct 2019 11:34:49 +0000

In the world of web development, ensuring the security and privacy of user data is paramount. One of the key aspects of this is implementing robust user authorization systems. In this article, we will delve into the process of building a simple yet effective user authorization system for the Phoenix framework, a system that can be seamlessly integrated into applications of various sizes. Whether you're a seasoned Elixir developer or a beginner just getting started with Phoenix, this guide will provide you with a practical approach to implementing authorization in your applications.

Essentially, authorization allows a user or an entity to perform an action if they have the necessary permissions. There are multiple approaches to address this. But this one matches my requirements and it allows me to easily apply this logic between multiple projects. As usual there are libraries to do this. But in my case it was not necessary.

Let's assume we want to authorize a rule where 'only the user who created a post can delete the post'. Basically, We can simply create a function to do that and call it from the action accessing it.

def authorize_delete_post(user, post) do
  if(user.id == post.user_id) do
    {:ok}
  else
   {:unauthorized}
  end
end

This is the simplest type of authorization. It will work fine, but if you're reading this, you're likely seeking a more robust solution.

Authorize module

It is recommended to centralize all authorization logic. So, we can create a new module Authorize and move it there.

defmodule MyApp.Authorize do

alias MyApp.Accounts.User
alias MyApp.Posts.Post

 def check(:delete_post, %User{id: user_id}, %Post{user_id: user_id}), do: {:ok}
 def check(:delete_post, %User{}, %Post{}), do: {:unauthorized}
end

From now on, we can call Authorize.check(:delete_post, user, post) to authorize the user to delete the post. it will return {:ok} if user_id of post is equal to id of user.

We can refactor here, as there will be a need to authorize many more actions. But remember, this part should be done based on your application's requirements.

defmodule MyApp.Authorize

alias MyApp.Accounts.User
alias MyApp.Posts.Post

 def check(:delete_post, %User{}=user, %Post{}=post), do: authorize_created_by(user, post)


 defp authorize_created_by(%User{id: user_id}, %{user_id: user_id}), do: ok
 defp authorize_created_by(%User{}, _), do: unauthorized

 defp ok, do: {:ok}
 defp unauthorized, do: {:unauthorized}
end

we can latter add

def check(:update_post, %User{}=user, %Post{}=post), do: authorize_created_by(user, post)

to do authorization for updating the post.

That concludes the implementation part. Now, we can optimize on the controller end.

Sending common error messages upon authorization error.

we have a plug in phoenix controller called action_fallback that allows us to call another plug as a fallback action. Follow the examples from https://hexdocs.pm/phoenix/Phoenix.Controller.html#action_fallback/1 to understand how to do that. I don't want to cover it here since it is already well documented. Also as a bonus you have information on how you should call our authorization module in there. So, in our case the FallbackController will have this plug.

def call(conn, {:unauthorized}) do
  # do something and return 403
end

Conclusion

In conclusion, implementing a user authorization system in Elixir and Phoenix doesn't have to be a daunting task. With the right approach and understanding, you can build a system that is not only simple and easy to read, but also easy to implement and optimize based on your application's business logic. The method we've explored in this article provides a solid foundation, but remember, the world of web development is always evolving. Always be open to exploring new techniques, libraries, and best practices to ensure your authorization system remains robust, secure, and efficient. Happy coding!

DEV Community: Haseeb Annadamban

SQL injection in Rails - Learn from an attacker’s shoes

Understanding SQL Injection

Be careful of the injection vector

Example 1: group method

Other methods similar to group

Example 2. calculate method

Misuse of the sanitize Method and Its Consequences

SQL injection can happen from stored data too

Passive detection of SQL injection

Additional level of defenses

Conclusion

Broken Access Control: What Is It and Why Does It Matter in your Rails application?

What is access control

What is broken access control

The Basics of access control

Authentication

Authorization

Access control implementation

Role Base Access Control (RBAC)

Others

Insecure Direct Object Access (IDOR)

Privilege Escalation

Horizontal privilege escalation

Vertical privilege escalation

Conclusion

Notes on Performance Optimization in Rails Applications

Database optimization

N+1 Query problem

The issue

The solution

Database indexing

Common query optimizations

select and pluck

Batch processing

You can avoid callbacks if you know what you are doing

Try to minimize db access

Counter cache

Caching

Pagination

Advanced: Read replicas

Memory optimization

Use latest stable version of ruby and rails

Utilize lazy loading when working with large collections

Use symbols instead of strings where appropriate

Limit session data

Use background jobs for long running processes

Use of correct algorithm

Use built-in ruby methods

Memoization in Rails

Avoiding nested loops

How to find what's the issue with a slow endpoint

Improving performance of specific slow SQL query.

Conclusion

How to run a ruby web server on your android device (without root).

Step 1 : Termux

Step 2 : Packages

Step 3 : Editor

Step 4 : Gems

Step 5 the server.rb

Step 6 : Visit the website

Step 7 Stopping the server

Now what

How to implement Authorization in elixir and phoenix

Authorize module

Sending common error messages upon authorization error.

Conclusion

Example 1: `group` method

Example 2. `calculate` method

`select` and `pluck`