How to Upgrade Rails Without a Test Suite

Sometimes you will be caught between a rock and a hard place. For example: You really need to upgrade your Rails application because Heroku is discontinuing your stack in two weeks so you don't have the time to write an entire test suite.

Sometimes the situation will be that your boss really needs you to upgrade Ruby or Rails to use a version that gets security updates and they won't allow you to write tests beforehand.

This article will explain how to ship a Rails upgrade project without a test suite. If your application has no tests, you will learn how to upgrade without having to write tests first.

1. Set up Staging
2. Check the Changelog
3. Write a Testing Script
4. Set up Error Tracking
5. Set up Dual Booting
6. Upgrade Development and Test
7. Deploy to Staging and Test
8. Deploy to Production
9. If you have the time, write integration tests
10. Summary

1. Set up Staging

Have a staging server that is identical to your production server. You will need to thoroughly test your changes in a staging environment before you deploy any changes to production.

Strive to make this environment as similar to production as possible. Fill it up with production-like data, so that you can test real world scenarios.

2. Changelog

Many of the changes in the new Rails version won't affect your application. Read the most notable changes in the changelog and manually find whether the APIs that changed are getting used in your application's code.

In this step there will be a lot of grepping or finding in files within your code editor. If you want to learn how to do this, we wrote an article about that: How we upgrade Rails applications

3. Write a Testing Script

If you know the application from top to bottom, write down several manual test scenarios. If you don't, work with the product owner to define a set of steps that you can run to smoke test the application in your development, staging, and production environments.

4. Set up Error Tracking

If you don't have this configured, make sure that you set it up before you deploy anything to staging or production.

There are many options when it comes to error tracking. Some of the most popular alternatives are Sentry; Honeybadger; or Rollbar. This will come in handy when you deploy your code and you get real users using your application on the new version of Rails.

Considering your application does not have a test suite, I can guarantee that you will get exception notifications once your code is live. You want to make sure that you can get all the necessary information so that you can quickly patch those bugs.

Even applications with a solid test suite and great code coverage will find exceptions in production. So this is a great idea even if you are not deploying a major Rails upgrade. :)

5. Set up Dual Booting

This will help you with the next steps across all your environments. Using this technique you will save a lot of time, because you will be able to run your application with Rails X or Y by changing a single environment variable.

We have explained how to dual boot in this article:How to Dual Boot Ruby on Rails

6. Upgrade Development and Test

If you have set up dual booting correctly, you will be able to run your current Rails version like this:

bundle exec rails server

And your new Rails version like this:

next bundle exec rails server

Now you can start smoke testing your application in your development environment. This step will include a lot of steps that look like this:

1. Read and execute your testing script
2. Monitor your logs and error tracking service
3. Find issues
4. Patch them
5. And repeat...

Once you feel comfortable that you have tested your critical user paths and everything looks good, you can move on to deploying your changes to your staging environment.

7. Deploy to Staging and Test

Now you are at a point where you can involve your QA team. Once you have deployed and tested your changes in staging, you can involve more people in helping you manually test critical user flows.

8. Deploy to Production and Test

There are two safe-ish ways to deploy your Rails upgrade to production:

1. Gradual deployment
2. Full deployment

Gradual deployment

This is another step that benefits from your dual booting code. You can deploy your changes to only a few application instances in your cluster. Then you will have your cluster running both versions in parallel (Rails X and Y).

You can set up your production environment to direct only 10% of the traffic to the new version of your code. This article explains how you can do that:Continuously Integrating a Major Rails Upgrade

Full deployment

If your application does not get a lot of traffic at night, you might want to deploy your changes to all your visitors then. This approach is certainly riskier because slower traffic means fewer users which means fewer "testers" for real world scenarios.

In both scenarios, you will eventually sit and wait for errors. Depending on the severity of the errors, you might want to roll back your changes and go back tostep 6.

If there are only a few minor bugs, you might want to hot fix them in production.

9. Integration Tests

If you have the time, I would highly recommend writing some integration tests. You can use RSpec and Capybara to write some simple tests to view them in a browser.

You don't have to get too creative when writing these tests, you can basically write something like this:

# spec/features/homepage_spec.rb
require "rails_helper"

RSpec.describe "Homepage at FastRuby.io", js: true do
  it "does not blow up" do
    visit "/"

    expect(page).to have_text("Need to upgrade Rails but don't have the time?")
  end
end

You can write basic, smoke-testy integration tests that mimic the behavior of end users and get to a non-zero amount of code coverage in a few minutes. If you have the time, doing this will save you time in your workflow.

Summary (TL;DR)

Someone is going to test your application. It could be you, writing automated tests and increasing code coverage. It might be your QA team, testing things work as expected in staging and production. In the worst case, it will be your end users trying to do something in production.

If you don't have a test suite, it does not mean that you cannot upgrade a dependency of your application. It usually means that it will take longer than if you had a well-written test suite, because there will be a lot of manual testing during the project.