To give back to the developer community, at Rollbar we looked at our database of thousands of projects and found the top 10 errors in Ruby on Rails projects. We’re going to show you what causes them and how to prevent them from happening. If you avoid these "gotchas," it'll make you a better developer.
Because data is king, we collected, analyzed, and ranked the top 10 Ruby errors from Ruby on Rails applications. Rollbar collects all the errors for each project and summarizes how many times each one occurred. We do this by grouping errors according to fingerprinting. Basically, we group two errors if the second one is just a repeat of the first. This gives users a nice overview instead of an overwhelmingly big dump like you’d see in a log file.
We focused on the errors most likely to affect you and your users. To do this, we ranked errors by the number of projects experiencing them across different companies. We intentionally looked at the number of projects so that high-volume customers wouldn't overwhelm the data set with errors that are not relevant to most readers.
Here are the top 10 Rails errors:
You’ve probably noticed some familiar faces in there already. Let’s dig in and take a look at the errors in a bit more detail to see what might cause them in your production application.
We'll provide example solutions based on Rails 5, but if you’re still using Rails 4 they should point you in the right direction.
1. ActionController::RoutingError
We start with a classic of any web application, the Rails version of the 404 error. An ActionController::RoutingError means that a user has requested a URL that doesn’t exist within your application. Rails will log this and it will look like an error, but for the most part it is not the fault of your application.
It may be caused by incorrect links pointing at or from within your application. It may also be a malicious user or bot testing your application for common weaknesses. If that’s the case, you might find something like this in your logs:
ActionController::RoutingError (No route matches [GET] "/wp-admin"):
There is one common reason you might get an ActionController::RoutingError that is caused by your application and not by errant users: if you deploy your application to Heroku, or any platform that doesn’t allow you to serve static files, then you might find that your CSS and JavaScript doesn’t load. If this is the case, the errors will look like this:
ActionController::RoutingError (No route matches [GET] "/assets/application-eff78fd93759795a7be3aa21209b0bd2.css"):
To fix this and allow Rails to serve static assets you need to add a line to your application’s config/environments/production.rb file:
Rails.application.configure do
# other config
config.public_file_server.enabled = true
end
If you aren’t interested in logging 404 errors caused by ActionController::RoutingError then you can avoid them by setting a catch all route and serving the 404 yourself. This method is suggested by the lograge project. To do so, add the following at the bottom of your config/routes.rb file:
Rails.application.routes.draw do
# all your other routes
match '*unmatched', to: 'application#route_not_found', via: :all
end
Then add the route_not_found method to your ApplicationController:
class ApplicationController < ActionController::Base
protect_from_forgery with: :exception
def route_not_found
render file: Rails.public_path.join('404.html'), status: :not_found, layout: false
end
end
Before implementing this, you should consider whether knowing about 404 errors is important to you. You should also keep in mind that any route or engine that is mounted after the application loads won’t be reachable as they will be caught by the catch all route.
2. NoMethodError: undefined method '[]' for nil:NilClass
This means that you are using square bracket notation to read a property from an object, but the object is missing, or nil, and thus it does not support this method. Since we are working with square brackets, it’s likely that we’re digging through hashes or arrays to access properties and something along the way was missing. This could happen when you’re parsing and extracting data from a JSON API or a CSV file, or just getting data from nested parameters in a controller action.
Consider a user submitting address details through a form. You might expect your parameters to look like this:
{ user: { address: { street: '123 Fake Street', town: 'Faketon', postcode: '12345' } } }
You might then access the street by calling params[:user][:address][:street]. If no address was passed then params[:user][:address] would be nil and calling for [:street] would raise a NoMethodError.
You could perform a nil check on each parameter and return early using the && operator, like so:
street = params[:user] && params[:user][:address] && params[:user][:address][:street]
While that will do the job, thankfully there is now a better way to access nested elements in hashes, arrays and event objects like ActionController::Parameters. Since Ruby 2.3, hashes, arrays and ActionController::Parameters have the dig method. dig allows you to provide a path to the object you want to retrieve. If at any stage nil is returned, then dig returns nil without throwing a NoMethodError. To get the street from the parameters above you can call:
street = params.dig(:user, :address, :street)
You won't get any errors from this, though you do need to be aware that street may still be nil.
As an aside, if you are also digging through nested objects using dot notation, you can do this safely in Ruby 2.3 too, using the safe navigation operator. So, rather than calling
street = user.address.street
and getting a NoMethodError: undefined method street for nil:NilClass, you can now call.
street = user&.address&.street
The above will now act the same as using dig. If the address is nil then street will be nil and you will need to handle the nil when you later refer to the street. If all the objects are present, street will be assigned correctly.
While this suppresses errors from being shown to the user, if it still impacts user experience, you might want to create an internal error to track either in your logs or in an error tracking system like Rollbar so you have visibility to fix the problem.
If you are not using Ruby 2.3 or above you can achieve the same as above using the ruby_dig gem and ActiveSupport's try to achieve similar results.
3. ActionController::InvalidAuthenticityToken
Number 3 on our list requires careful consideration as it is related to our application's security. ActionController::InvalidAuthenticityToken will be raised when a POST, PUT, PATCH, or DELETE request is missing or has an incorrect CSRF (Cross Site Request Forgery) token.
CSRF is a potential vulnerability in web applications in which a malicious site makes a request to your application on behalf of an unaware user. If the user is logged in their session cookies will be sent along with the request and the attacker can execute commands as the user.
Rails mitigates CSRF attacks by including a secure token in all forms that is known and verified by the site, but can't be known by a third party. This is performed by the familiar ApplicationController line
class ApplicationController < ActionController::Base
protect_from_forgery with: :exception
end
So, if your production application is raising ActionController::InvalidAuthenticityToken errors it could mean that an attacker is targeting the users of your site, but the Rails security measures are keeping you safe.
There are other reasons you may be unintentionally receiving this error though.
Ajax
For example, if you are making Ajax requests from your front end, you need to ensure you are including the CSRF token within the request. If you are using jQuery and the built in Rails unobtrusive scripting adapter then this is already handled for you. If you want to handle Ajax another way, say using the Fetch API, you'll need to ensure you include the CSRF token. For either approach, you need to make sure your application layout includes the CSRF meta tag in the <head> of the document:
<%= csrf_meta_tags %>
This prints out a meta tag that looks like this:
<meta name='csrf-token' content='THE-TOKEN'>
When making an Ajax request, read the meta tag content and add it to the headers as the X-CSRF-Token header.
const csrfToken = document.querySelector('[name="csrf-token"]').getAttribute('content');
fetch('/posts', {
method: 'POST',
body: JSON.stringify(body),
headers: {
'Content-Type': 'application/json'
'X-CSRF-Token': csrfToken
}
).then(function(response) {
// handle response
});
Webhooks/APIs
Sometimes there are valid reasons to turn off the CSRF protection. If you expect to receive incoming POST requests to certain URLs in your application from third parties, you won’t want to block them on the basis of CSRF. You might be in this position if you are building an API for third party developers or if you expect to receive incoming webhooks from a service.
You can turn off CSRF protection, but make sure you are whitelisting the endpoints you know don't need this kind of protection. You can do so in a controller by skipping the authentication:
class ApiController < ApplicationController
skip_before_action :verify_authenticity_token
end
If you are accepting incoming webhooks, you should be able to verify that the request came from a trusted source in place of verifying the CSRF token.
4. Net::ReadTimeout
The Net::ReadTimeout is raised when it takes Ruby longer to read data from a socket than the read_timeout value, which is 60 seconds by default. This error can be raised if you are using Net::HTTP, open-uri or HTTParty to make HTTP requests.
Notably, this doesn't mean that an error will be thrown if the request itself takes longer than the read_timeout value, just that if a particular read takes longer than the read_timeout. You can read more about Net::HTTP and timeouts from Felipe Philipp.
There are a couple of things we can do to stop getting Net::ReadTimeout errors. Once you understand the HTTP requests that are throwing the error you can try to adjust the read_timeout value to something more sensible. As in the article above, if the server you are making the request to takes a long time to put together a response before sending it all at once, you will want a longer read_timeout value. If the server returns the response in chunks then you will want a shorter read_timeout.
You can set read_timeout by setting a value in seconds on the respective HTTP client you are using:
with Net::HTTP
http = Net::HTTP.new(host, port, read_timout: 10)
with open-uri
open(url, read_timeout: 10)
with HTTParty
HTTParty.get(url, read_timeout: 10)
You can't always trust another server to respond within your expected timeouts. If you can run the HTTP request in a background job with retries, like Sidekiq, that can mitigate the errors from the other server. You will need to handle the case where the server never responds in time though.
If you need to run the HTTP request within a controller action, then you should be rescuing the Net::ReadTimeout error and providing your user with an alternative experience and tracking it in your error monitoring solution. For example:
def show
@post = Post.find(params[:slug])
begin
@comments = HTTParty.get(COMMENTS_SERVER, read_timeout: 10)
rescue Net::ReadTimeout => e
@comments = []
@error_message = "Comments couldn't be retrieved, please try again later."
Rollbar.error(e);
end
end
5. ActiveRecord::RecordNotUnique: PG::UniqueViolation
This error message is specifically for PostgreSQL databases, but the ActiveRecord adapters for MySQL and SQLite will throw similar errors. The issue here is that a database table in your application has a unique index on one or more fields and a transaction has been sent to the database that violates that index. This is a hard problem to solve completely, but let's look at the low hanging fruit first.
Imagine you've created a User model and, in the migration, ensured that the user's email address is unique. The migration might look like this:
class CreateUsers < ActiveRecord::Migration[5.1]
def change
create_table :users do |t|
t.string :email
t.timestamps
end
add_index :users, :email, unique: true
end
end
To avoid most instances of ActiveRecord::RecordNotUnique you should add a uniqueness validation to your User model too.
class User < ApplicationRecord
validates_uniqueness_of :email
end
Without this validation, all email addresses will be sent to the database when calling User#save and will raise an error if they aren't unique. However, the validation can't guarantee that this won't happen. For a full explanation you should read the concurrency and integrity section of the validates_uniqueness_of documentation. The quick description is that the Rails uniqueness check is prone to race conditions based on the order of operation for multiple requests. Being a race condition, this also makes this error hard to reproduce locally.
To deal with this error requires some context. If the errors are caused by a race condition, that may be because a user has submitted a form twice by mistake. We can try to mitigate that issue with a bit of JavaScript to disable the submit button after the first click. Something a bit like this is a start:
const forms = document.querySelectorAll('form');
Array.from(forms).forEach((form) => {
form.addEventListener('submit', (event) => {
const buttons = form.querySelectorAll('button, input[type=submit]')
Array.from(buttons).forEach((button) => {
button.setAttribute('disabled', 'disabled');
});
});
});
This tip on Coderwall to use ActiveRecord's first_or_create! along with a rescue and retry when the error is raised is a neat workaround. You should continue to log the error with your error monitoring solution so that you maintain visibility on it.
def self.set_flag( user_id, flag )
# Making sure we only retry 2 times
tries ||= 2
flag = UserResourceFlag.where( :user_id => user_id , :flag => flag).first_or_create!
rescue ActiveRecord::RecordNotUnique => e
Rollbar.error(e)
retry unless (tries -= 1).zero?
end
ActiveRecord::RecordNotUnique might seem like an edge case, but it's here at number 5 in this top 10, so it is definitely worth considering with regard to your user experience.
6. NoMethodError: undefined method 'id' for nil:NilClass
NoMethodError appears again, though this time with a different explanatory message. This error usually sneaks up around the create action for an object with a relation. The happy path—creating the object successfully—usually works, but this error pops up when validations fail. Let's take a look at an example.
Here's a controller with actions to create an application for a course.
class CourseApplicationsController < ApplicationController
def new
@course_application = CourseApplication.new
@course = Course.find(params[:course_id])
end
def create
@course_application = CourseApplication.new(course_application_params)
if @course_application.save
redirect_to @course_application, notice: 'Application submitted'
else
render :new
end
end
private
def course_application_params
params.require(:course_application).permit(:name, :email, :course_id)
end
end
The form in the new template looks a bit like this:
<%= form_for [@course, @course_application] do |ca| %>
<%# rest of the form %>
<% end %>
The problem here is when you call render :new from the create action, the @course instance variable wasn't set. You need to ensure that all the objects the new template needs are initialised in the create action as well. To fix this error, we'd update the create action to look like this:
def create
@course_application = CourseApplication.new(course_application_params)
if @course_application.save
redirect_to @course_application, notice: 'Application submitted'
else
@course = Course.find(params[:course_id])
render :new
end
end
Check out this article if you are interested in learning more about the problems with nil in Rails and how to avoid them.
7. ActionController::ParameterMissing
This error is part of the Rails strong parameters implementation. It does not manifest as a 500 error though—it is rescued by ActionController::Base and returned as a 400 Bad Request.
The full error might look like this:
ActionController::ParameterMissing: param is missing or the value is empty: user
This will be accompanied by a controller that might look a bit like this:
class UsersController < ApplicationController
def create
@user = User.new(user_params)
if @user.save
redirect_to @user
else
render :new
end
end
private
def user_params
params.require(:user).permit(:name, :email)
end
end
The params.require(:user) means that if user_params is called and params does not have a :user key or params[:user] is empty, ActionController::ParameterMissing will be raised.
If you are building an application to be used via a web front end and you have built a form to correctly post the user parameters to this action, then a missing user parameter probably means someone is messing with your application. If that is the case, a 400 Bad Request response is likely the best response as you don't need to cater to potentially malicious users.
If your application is providing an API, then 400 Bad Request is also an appropriate response to a missing parameter.
8. ActionView::Template::Error: undefined local variable or method
This is our only ActionView error in the top 10 and that's a good sign. The less work the views have to do to render templates the better. Less work leads to fewer errors. We’re still left with this error though, in which a variable or method you expect to exist simply doesn't.
This crops up most commonly in partials, probably due to the many different ways you can include a partial with local variables on a page. If you have a partial called _post.html.erb that contains a blog post template and an instance variable @post set in your controller, then you can render the partial like this:
<%= render @post %>
or
<%= render 'post', post: @post %>
or
<%= render partial: 'post', locals: { post: @post } %>
Rails likes to give us plenty of options to work with, but the second and third options here are where confusion can creep in. Trying to render a partial like:
<%= render 'post', locals: { post: @post } %>
or
<%= render partial: 'post', post: @post %>
will leave you with an undefined local variable or method. To avoid this, stay consistent and always render partials with the explicit partial syntax, expressing the local variables in a locals hash:
<%= render partial: 'post', locals: { post: @post } %>
There is one other place you can slip up with local variables in partials. If you only sometimes pass a variable to a partial, testing for that variable is different within a partial to regular Ruby code. If, for example, you update the post partial above to take a local variable that tells you whether to show a header image in the partial, you would render the partial like so:
<%= render partial: 'post', locals: { post: @post, show_header_image: true } %>
Then the partial itself might look like this:
<h1><%= @post.title %></h1>
<%= image_tag(@post.header_image) if show_header_image %>
<!-- and so on -->
This will work fine when you pass the show_header_image local variable, but when you call
<%= render partial: 'post', locals: { post: @post } %>
it will fail with an undefined local variable. To test for the existence of a local variable inside a partial, you should check whether it is defined before you use it.
<%= image_tag(@post.header_image) if defined?(show_header_image) && show_header_image %>
Even better though, there is a hash called local_assigns within a partial that we can use instead.
<%= image_tag(@post.header_image) if local_assigns[:show_header_image] %>
For variables that aren't booleans, we can use other hash methods like fetch to handle this gracefully. Using show_header_image as an example, this scenario would also work:
<%= image_tag(@post.header_image) if local_assigns.fetch(:show_header_image, false) %>
Overall, watch out when you are passing variables to partials!
9. ActionController::UnknownFormat
This error, like the ActionController::InvalidAuthenticityToken, is one that could be caused by careless or malicious users rather than your application. If you've built an application in which the actions respond with HTML templates and someone requests the JSON version of the page, you will find this error in your logs, looking a bit like this:
ActionController::UnknownFormat (BlogPostsController#index is missing a template for this request format and variant.
request.formats: ["application/json"]
request.variant: []):
The user will receive a 406 Not Acceptable response. In this case they’ll see this error because you haven't defined a template for this response. This is a reasonable response, since if you don't want to return JSON, their request was not acceptable.
You may, however, have built your Rails application to respond to regular HTML requests and more API-like JSON requests in the same controller. Once you start doing this, you define the formats you do want to respond to and any formats that fall outside of that will also cause an ActionController::UnknownFormat, returning a 406 status. Let’s say you have a blog posts index that looks like:
class BlogPostsController < ApplicationController
def index
respond_to do |format|
format.html { render :index }
end
end
end
Making a request for the JSON would result in the 406 response and your logs would show this less expressive error:
ActionController::UnknownFormat (ActionController::UnknownFormat):
The error this time doesn't complain about a lack of a template—it’s an intentional error because you have defined the only format to respond to is HTML. What if this is unintentional though?
It’s common to miss a format in a response that you intend to support. Consider an action in which you want to respond to HTML and JSON requests when creating a blog post, so that your page can support an Ajax request. It might look like this:
class BlogPostsController < ApplicationController
def create
@blog_post = BlogPost.new(blog_post_params)
respond_to do |format|
if @blog_post.save
format.html { redirect blog_post_path(@blog_post) }
format.json { render json: @blog_post.to_json }
else
render :new
end
end
end
end
The error here is raised in the case of the blog post failing validations and not saving. Within the respond_to block, you need to call render within the scope of the format blocks. Rewriting this to accommodate for failure would look like:
class BlogPostsController < ApplicationController
def create
@blog_post = BlogPost.new(blog_post_params)
respond_to do |format|
if @blog_post.save
format.html { redirect blog_post_path(@blog_post) }
format.json { render json: @blog_post.to_json }
else
format.html { render :new }
format.json { render json: @blog_post.errors.to_json }
end
end
end
end
Now all of the formats are covered and there won't be any more unintentional ActionController::UnknownFormat exceptions.
10. StandardError: An error has occurred, this and all later migrations canceled
This last item on our top 10 disappoints me slightly. StandardError is the base error class that all other errors should inherit from, so using it here makes the error feel very generic, when in reality it is an error that has happened during a database migration. I would prefer to see this error as a descendent of the ActiveRecord::MigrationError. But I digress…
There are a number of things that can cause a migration to fail. Your migrations may have gotten out of sync with your actual production database, for example. In that case, you're going to have to go digging around to find out what has happened and fix it.
There is one thing that should be covered here though: data migrations.
If you need to add or calculate some data for all the objects in a table you might think that a data migration is a good idea. As an example, if you wanted to add a full name field to a user model that included their first and last name (not a likely change, but good enough for a simple example), you might write a migration like this:
class AddFullNameToUser < ActiveRecord::Migration
def up
add_column :users, :full_name, :string
User.find_each do |user|
user.full_name = "#{user.first_name} #{user.last_name}"
user.save!
end
end
def down
remove_column :users, :full_name
end
end
There are a lot of problems with this scenario. If there is a user with corrupt data in y x xour set, the user.save! command will throw an error and cancel the migration. Secondly, in production you may have a lot of users, which means the database would take a long time to migrate, possibly keeping your application offline for the entire time. Finally, as your application changes over time, you might remove or rename the User model, which would cause this migration to fail. Some advice suggests that you define a User model within the migration to avoid this. For even greater safety, Elle Meredith advises us to avoid data migrations within ActiveRecord migrations completely and build out temporary data migration tasks instead.
Changing data outside of the migration ensures you do a few things. Most importantly, it makes you consider how your model works if the data is not present. In our full name example, you would likely define an accessor for the full_name property that could respond if the data was available. If it’s not, then build the full name by concatenating the constituent parts.
class User < ApplicationRecord
def full_name
@full_name || "#{first_name} #{last_name}"
end
end
Running a data migration as a separate task also means the deploy no longer relies on this data changing across the production database. Elle's article has more reasons why this works better and includes best practices on writing the task as well.
Conclusion
The most popular Rails errors can come from anywhere within the application. In this article we've seen common errors manifested in the model, the view, and the controller. Some of the errors aren't necessarily anything to worry about and are just protecting your application. Others should be caught as soon as possible and stamped out.
Nevertheless, it’s good to track how often these errors happen. This leads to better visibility into problems that are affecting your users or application security, so you can fix them quickly. Otherwise, these error messages will be shown to the user, but the engineering and product management teams will have no idea until users complain to support.
We hope you learned something new and are better equipped to avoid these errors in the future. However, even with the best practices, unexpected errors do pop up in production. It's important to have visibility into errors that affect your users, and to have good tools to solve them quickly.
Rollbar gives you visibility to production Ruby errors, which offers more context to solve them quickly, including form validation errors, person tracking, boot errors and more. Check out Rollbar’s full list of features and Ruby SDK documentation.
Top 10 errors from 1000+ Ruby on Rails projects (and how to avoid them) was originally published on the Rollbar blog on April 18, 2018.
Top comments (11)
Thank you for including the "...and how to avoid them" part. Awesome!
That's the most important part! Glad you enjoyed it!
Right away I saw number #1 and I was I thought of my rollbar being filled the brim which these reports.
So at the application level is one way to filter them ou as shown here for unmatched routes.
The cloud computing way would be to filter them out with your Web Application Firewall (WAF), so I do this using AWS WAF.
The way I can keep my application as vanilla as possible and if I wanted to I can now automate a response from based on the behaviour of these missing routes since sometimes they are malicious.
I have to do this because people attempt to scrape my paid content and so I have the ability to detect and honey pot these users. Could you do this within your Rails app? yeah but I don't want that traffic even making it to my instances.
Phil: I would love to know when Twillio will support Twillo SIM cards in Canada. I was lucky enough o be lent a Twilio IoT kit from a Twillio Champion and I was going to put together a free IoT workshop but sadly the SIM cards are not yet supported over border.
Hey Andrew,
That's a good point about filtering out the routing errors before they even hit the application. I've not used a WAF before, have you written up anything on how to do this with a Rails application (or can you point me somewhere for that)? I definitely appreciate keeping the useless or malicious traffic away from the app itself.
As for Twilio SIM support, the narrowband SIM you have works on T-Mobile's NB-IoT network in the US. That's the only narrowband partnership we have right now and I don't know anything more than that.
We do have regular Twilio Wireless SIMs that are great for building IoT devices too. They work all around the world. We also have the Twilio Super SIM coming soon that will switch networks (rather than roaming) and will also work globally. Would you be interested in working with these at all?
Hey Phil,
I do have AWS WAF video tutorials but I have not had time to publish them.
For AWS WAF they have a WAF marketplace where you can purchase WAF rules that will filter them out for you. So AWS WAF I believe can cost around $7 per / month. The purchased rules from a vendor vary but can be additional $20 on top per month.
To be cost-effective you can just write your own rules in AWS WAF and this is what I do. You can add rules based on regex patterns and so I just look in my rollbar for bizarre routes with bizarre User Agents and I add them to my AWS WAF ruleset. I also prefer this approach because I get to familiarize myself with the kind of traffic.
AWS has a free CloudFormation in the AWS Docs that sets up a honey pot for you. I've modified this template for myself to deal with and be alerted of scrapers.
Thank you for sharing more information on the variety of SIMs.
Wireless and Super SIM I didn't investigate so maybe there is something feasible here.
I appreciate you dug that up for me.
I hope to never run into these errors, but just in case, I'm bookmarking this link and sharing it with my cohort. :)
Thanks for the detailed write-up.
If you never get a 404 in production I'd be amazed! 😄 Just make sure you have this list on hand so that you don't panic.
Hope it helps you and your whole cohort!
However I am a rails developer for more than 11 years , but you gave me tons of new informations
That's awesome, thanks for sharing, I'm glad I could help!
My top 1 is ActiveRecord::RecordNotFound
Ah, another 404 sort of error. Is there something you're doing to cause that do you think? Links that are pointed to old, deleted resources, or something like that?