DEV Community: Aaron Sumner

Getting started with a legacy Rails application

Aaron Sumner — Fri, 15 May 2020 03:21:52 +0000

This post originally appeared in Everyday Rails, about using the Ruby on Rails web application framework to get stuff done as a web developer.

After nearly seven years of working primarily in one Rails codebase in my day job, I've recently transitioned to a handful of other Rails applications, all written by different teams, at different times, and with different understandings of the framework. And that code I started working with seven years ago had been in active development for several years before I arrived, so it had its own long, decorated history.

Chances are that if you start a new Rails job, it's going to be to work with an established application with a lot of history, too. The label thrown around for such applications is often legacy. That term means different things to different people, but for this article, we'll focus on one characteristic: context. Legacy software projects are those for which a current developer lacks full knowledge of its context, yet the software is still actively used and relied upon. (I did not come up with that loose definition, but I'm having trouble tracking down who did--if you know, please let me know.)

Working with legacy codebases can be daunting, especially that first time you clone a repository down to your development workstation and fire up your editor. But as already mentioned, legacy applications still have users relying on them to perform. They have lots to teach us as developers, and give us opportunities to make improvements where we may not have been able to with a pristine application. Knowing how to work with these codebases is a valuable skill.

In this article, I'll share where I look for clues about the application's purpose, its implementation, and how to be productive as its caregiver going forward.

README

This might seem obvious, but the first place I'll check in the new (to me) application is its README, asking the following questions:

Does it even have one? All the applications I recently picked up have README files, but I've run into some in the past that either have none, or have the default file generated by the rails new command.

If your application doesn't have a README, your first task is to create one, and fill in the bits of information you do know about the application so far. For now, that might only be its name and a little bit of its business purpose, but that's fine! It's more than what was there before.

Does it include development setup instructions? Just note whether or not it does, for now. They may or may not be accurate. Depending on if the application has been maintained recently or not, you may need to revise whatever's here. More on this a little later.

Gemfile

You can learn a lot about what you've gotten yourself into by looking at a Rails application's Gemfile. It's the next file I'll check in the codebase, looking for the following:

Which Rails version does the app use? Luckily, all the applications I just picked up use version 4.0 or newer. Version 3.x Rails apps are still out there doing their thing, too. Have you been requested to work with something even older than that?
Which Ruby version was the application built on? This detail is sometimes included in the Gemfile, especially if the application is deployed to Heroku. If you don't see it, check for a .ruby-version file. If that's missing as well, you may be able to track down more information by looking at the application's production environment. Which Ruby is in use on the server hosting the software?
What are the app's dependencies? Some may be familiar, some not. I like to search RubyGems for those unfamiliar ones, to get a sense of why they're included, if they're still in active development, and general popularity. I do not judge a given gem's merits on either of those last two attributes, but they're still good to know.
Does the application use a typical authentication library? For Rails, this is often Devise. But it could be OmniAuth, or Clearance, or a handful of other gems. A lack of one of these included as a dependency may be a sign that the application uses a custom mechanism for authentication, or has none at all. Neither of these is necessarily a red flag, but it is something to watch for moving forward into development. Consider performing a security audit to ensure login isn't susceptible to attacks, early on.

If you don't even see a Gemfile, you may be working with a really old application that hasn't gotten any attention in some time. Bundler support wasn't formally added to Rails until version 3.0. Your life may have just become a lot more interesting. You might need to do some extra work to get older Rubies and dependencies up and running for development.

Application structure

Next, I'll start looking at things that make the application unique. While it's tempting to start diving deep here, I like to continue at a high level for now. If the app follows Rails conventions, I can learn a lot by exploring:

config/routes.rb: Does the application have complex routes with heavy nesting? Does it lean on RESTful resources? Does it have lots of controllers, or fewer, potentially overworked controllers?
rails routes: An alternate view of the app's routes, their structure, and any helper methods available to make accessing them easier.
app/models: How many models are present in this directory? Do their names make sense in the context of the application's business purpose, as you understand it at this point? I don't dig into the files themselves yet, but I do try to get a rough count of how many models there are, and rough file sizes for each--theoretically, more complex applications have more models. Large files may likely be the application's god objects. Make notes for future exploration!
rails stats: Finally, we can get a nice, broad strokes report of the application structure by checking its stats and asking questions like, how many lines of code are in models, relative to controllers, relative to libraries? Where's the test coverage--in low level unit tests, or high level system or integration tests, or somewhere in the middle? (More on test coverage later.) And how many methods per class (M/C) and lines of code per method (LOC/M) are there on average? While the numbers represent averages, they tend to help show how past developers structured code, so you can begin knowing where to look for more details.

Data schema

The application's database schema often sheds additional light on its complexity. Take a couple of minutes to skim db/schema.rb (assuming it's a typical Active Record-based app) to get a sense of its table structure. Does it have lots of tables, or just a few? Do any tables look like they're doing too much? Does the application leverage indexing or other database layer-specific features?

As long as you're in the directory, take a look to see if db/seeds.rb exists, and has anything in it beyond the commented sample code provided by Rails. Does it look like it'll help you set up your development environment later? If not, do you know what'll be involved in getting accounts or other required data in place to use the app locally?

Assets

Rails has a long, complicated history with Javascript. Depending on your application's age, don't be surprised to see jQuery, or Backbone, or Prototype, or React, or Coffeescript--or some combination of them! The same applies to styling; the app may use vanilla CSS, or Sass, or some other approach to building out stylesheets.

On initial exploration, the keys to watch for are:

Which frameworks are used?
Where are the assets stored--in app/assets, in public, in gems, inlined in a view, or from an externally hosted source referenced in a view or layout? Or it could have a bespoke system for injecting dynamic Javascript from controllers into views, like the application I recently stopped working on! (Probably not, but you never know.)
How are assets compiled for production (or are they at all)? For a long time, this has typically been handled at least in part by the asset pipeline in Rails, but be prepared for other approaches in both really old applications and more recent ones.

External dependencies

A Rails application isn't just Ruby code alone. There are external factors like the database, perhaps a separate key/value store for things like caching and background process management, mail delivery, file storage, image manipulation, and the list goes on. Can these be gleaned from the README, or the Gemfile, or container orchestration configuration, or some other documentation? Do they use default settings, or is customization required? Do developers manage these externalities, or will you need to reach out to another team for support?

There are also external services interfaced via APIs to consider. What are these services? Are there credentials required to access them? How are they configured? Are they different between development, testing, and production environments? Are there rate limits or other restrictions to be aware of as a developer? Not only can awareness of these factors help you get started coding on the new-to-you project, it can also help avoid taking out production systems by accident!

Version control history

I'm a big fan of learning about a project through its version control history. When was its first commit? (git log -p --reverse). When was its last commit? (git log -p). How are commits structured--as small, atomic changes, or sweeping overhauls? Do commits include detailed messages describing the need and intent behind the code change?

If you're lucky, your inherited codebase's previous developers practiced good version control hygiene, and the application's history tells a story that guides future work. If not, the history can still help identify past contributors, and key points in the project's history such as major upgrades or other big changes. If the project's hosted on GitHub or another platform that provides commit analytics, refer to them as well for additional insight.

Code documentation

This is a bit of a reach for most Ruby projects, especially a Rails application, but I do like to see if previous developers have written any documentation for the code, especially if it's light on unit testing. Before version 5, Rails included a Rake task (rake doc:app) to output docs into HTML. This feature has since been removed, but can be added back with the sdoc gem.

As an aside, I've begun adding documentation to my applications as I learn about them. This is especially useful for apps with models that may make perfect sense to people who've been thinking in the domain they represent for years, but are totally foreign to newcomers. It helps solidify my understanding of the application, and provides information for future developers.

Set up for development

With all the background knowledge in place, let's see if it's enough to get started working on this new-to-you codebase. Can you get the app to boot for development? I've run into problems with apps on really old versions of Ruby running into issues installing gems, or even Ruby itself, on newer versions of my development computer's operating system and system libraries. If that's the case for you, consider advocating a Ruby upgrade for you app, in the name of security and enabling feature enhancements. In the meantime, you could also set up a development environment using Docker, to help avoid these installation issues.

If there were setup instructions provided, are they correct, or did you have to make adjustments? If you did, create a new git branch with any changes you had to make to these instructions--even if you think nobody else is ever going to work with the code.

If you can get the Rails console to boot, and you can see your app running on localhost, congratulations! You're ready to work with your legacy application.

Test suite

For the apps I'm working with, working test suites have been more like nice bonuses than hard requirements. But it's worth the time to check to see the test suite's status. Some things to check:

What framework is the suite written in--MiniTest, RSpec, Test::Unit, Cucumber, or something else?
Does the suite run? And if it does, does it pass? Is it fast or slow?
Where's the coverage? You may get a sense of this by looking at the file structure in the test, spec, and/or features directories. Run rake stats if you haven't already, to get a rough idea. While you're at it, spot-check to see whether the files actually have any useful tests in them, or if they're boilerplate created by Rails generators--sadly, I have seen this when inheriting old codebases in the past, but luckily not this go-around.
If you noticed earlier that the application makes a lot of use of Javascript, is that functionality tested? Support for testing Javascript wasn't baked into Rails until relatively recently, so there's a decent chance that manual testing may be required in older apps. Just be prepared.

Summary

I hope this advice helps make your next foray into a legacy Rails codebase less scary. By digging into the areas I've listed here, you can begin to understand the application and its history, and equip yourself to make enhancements and upgrades as needed. Thank you for reading!

How to TDD when TDD is hard

Aaron Sumner — Sun, 14 Jul 2019 18:33:14 +0000

This post originally appeared in Everyday Rails. I write a lot about TDD there, so if you like this post, please check out Everyday Rails, too. Thanks!

When test-driven development works well, it's smooth and fluid, guiding you back and forth between test code and application code. It hints at what you need to do next—what to add, what to change, what to remove. And it does so quickly. The process is an asset, not a liability.

You and I both know it's not always an easy ride, though. Sometimes, TDD gets in the way, does more to confuse than guide, or generally just slows development to a monotonous crawl—leaving you staring repeatedly at vague test failures, or maybe just an empty line in your code editor, unsure where to even start.

When those times arise (and they will), here's a set of five go-to strategies I use to get out of test-fueled frustration, and back to productivity. While I mostly use them in Ruby on Rails with the RSpec testing framework, they're pretty general overall, and conceptually apply to TDD in other languages and frameworks.

Problem: This test isn't telling me anything

Whether you call it a system test, integration test, feature test, or acceptance test, a high-level test is often the starting point with TDD, especially if you practice outside-in testing like I generally do. This test is great for showing that the finished product works as intended, but is less great at actually driving development decisions. A failed system test's output is often too vague to point out problems in your app's deeper levels.

When a system test is too abstract to understand the problem, try testing at a lower level. In a stock Rails application, this could mean dropping down to a controller test, or even further down to a model test. If you've extracted service objects or similar, see if you can get meaning from tests that interact directly with them. If the test raised an exception instead of passing/failing, the resulting stack trace may help you decide where to test.

Ideally, code at lower levels requires fewer variables and conditions to understand its inner workings. Explore the individual objects expected to interact together to make a feature run, using tests as guides. See if you can test closer to the problematic code, then work your way back up toward a system test.

Problem: This test needs too much setup

If you're like me, you've got a love-hate relationship with mocking in tests. Most of the time, I like it just fine—until I run into a test that requires more and more to be stubbed out just to run. By the time I'm finished, my test's setup code outnumbers the actual test, ten to one. And the test is now so tightly coupled to the application code that any minor change to the latter ripples over to more and more test tweaks.

Sound familiar? I'm exaggerating slightly, but I think this is a common complaint about mocking. And I also think mocking gets a bad rap for it. The problem isn't always mocking—sometimes, it's a sign that the code under test has too much responsibility, and is a candidate for a refactor. I see this a lot in Rails controllers and their corresponding tests, especially in legacy apps that carry over pre-RESTful behavior.

When this occurs, try moving up to test from a higher level. On the downside, this new test may be slower than the test it's replacing. But the new test is often also easier to set up, and provides a safety net for refactoring. Use the high level test as a guide to help you break down that complicated code into smaller, more testable parts. Depending on the results of that exercise, you may find the high level test no longer necessary. Use your coverage metrics and professional judgment as guides.

Problem: This test is confusing

Many times, TDD requires you to dig into existing tests, rather than writing new ones. Maybe those old tests are supposed to tell you how the software works, or maybe they need to be adapted to support new behavior. But when an existing test is buried inside nested contexts, relies heavily on mystery guests, and is generally just overly refactored to the point that it's unreadable, it's only useful as safety net coverage for existing functionality. It can't evolve alongside your application.

I often see this in system tests, especially those written in RSpec. It can happen at other levels of testing, but the extra setup required by system tests seems to lend itself to over-refactoring. Nothing frustrates me more than a test that requires more time to understand than it takes to just add a trivial new application behavior!

When I run into a test like this, I try to tease it back into a decidedly un-DRY, single, inline test. Working in a new file often helps. Paste the confusing test's contents into the new file. Paste the setup from before blocks, nesting, and shared contexts that the confusing test relies on. Get the test to pass on its own, and then take some time to understand how it passes.

Once you've studied the test, you can either go forward with the original test, operating with a better understanding of how it works, or you can keep the new test and build upon it, dropping the original, confusing version. Again, you'll need to put some thought into which version is better. There's no one right way to do this sort of thing—and even when someone says there is, there are almost always exceptions to such rules.

Problem: This app doesn't have enough (or any!) test coverage to begin with

This is a tough one. I won't lie. But it's solvable, if you can put some time into it. I've successfully approached the problem of insufficient coverage, inspired by Katrina Owen's wonderful talk Therapeutic Refactoring. The trick is to assume existing behavior is correct, and use the code's current output to build out high-level coverage.

It's not always beautiful, but it'll get coverage in place for the features that matter most—and then you can use that coverage as a starting point to improve tests and the code being tested.

As a rule of thumb, you'll get broader coverage more quickly with high-level tests. Bear in mind, though, that focusing exclusively on this level to build out your coverage can quickly to a slow test suite, incapable of providing the rapid feedback necessary for TDD to flow smoothly. Use these new tests as a starting point, but try to limit them to critical use cases—and as early as possible, explore ways to speed up feedback loops by testing at lower, faster levels.

Problem: I have no idea what I'm doing

I'll admit, test-first, test-driven development isn't always as great as advertised. Sometimes, I can't even figure out what to test, or how to write it—never mind writing code to make it pass!

For these cases of tester's block, when a new feature's requirements don't immediately lend themselves to a test, the first thing I do is ask for clarification. I know that asking others for help can be difficult, but it's a a critical tool for developers at all career stages to master. Better to ask up front than make a faulty assumption!

If things are still unclear, I give myself permission to poke at the code a little bit, and get a better understanding of it. Maybe even try some ideas and see how they look in the browser (or UI, or API, or whatever your interface is). Break out the debugging tools (I like pry, but feel free to use a traditional debugger, or puts, or a mix). Usually, this gives me a sense of what to test. Experimenting with the code before writing tests can also guide where to start testing. Maybe a top-down testing approach doesn't apply to a certain situation, and starting at a lower level yields a better-crafted solution.

Problem: I just don't have time to TDD this!

Sometimes, deadlines are too pressing to ignore. When due dates loom, it's tempting to put testing aside, get that mission-critical feature out the door, and move on to the next deadline, with the pinky-swear promise that we'll go back and clean up that messy code, build out test coverage, and make things beautiful.

My experience tells me that all other TDD challenges we've talked about so far, lead up to this point. It's easy for me to say, well, make time! So I'm not going to say it, or at least, not in those words. Instead, let me frame the problem another way. Ask yourself a few questions: How confident are you in this code you're writing under pressure? What negative impact might this pressure have on your code? What's the worst thing that could happen if this code is wrong?

TDD is a double-layered safety net. Good tests guide good code. Yeah, sometimes, good code means good enough code—I'm not going to pretend to be above deadlines. And if and when you do have the time to go back and clean things up, a good enough test is the first step toward more performant, better-factored code.

Foregoing tests leads down a dangerous slope. I've seen this myself, with ignored test suites in business-critical software. As time passes and original developers move on (and they do), test suites get more and more difficult to resurrect. A neglected test suite is useless—and the code base it covers becomes the same in short order.

So do try to get at least some coverage in with every code change, even when you're short on time. That could be a single, high-level happy path test, or a few lower-level tests covering the important parts of an important algorithm. They may not be the prettiest tests, but that's OK! Increase reliability as you face the deadline at hand, and leave useful breadcrumbs for future development.

Replacing chromedriver-helper with webdrivers

Aaron Sumner — Wed, 10 Apr 2019 15:41:43 +0000

Originally posted in Everyday Rails.

chromedriver-helper is no longer supported. If you try to install it or upgrade to a newer version, you'll be greeted with this announcement:

+--------------------------------------------------------------------+
|                                                                    |
|  NOTICE: chromedriver-helper is deprecated after 2019-03-31.       |
|                                                                    |
|  Please update to use the 'webdrivers' gem instead.                |
|  See https://github.com/flavorjones/chromedriver-helper/issues/83  |
|                                                                    |
+--------------------------------------------------------------------+

As the message suggests, if you need to test JavaScript within the confines of a Rails app, then webdrivers is how you'll want to manage Selenium drivers. If you've purchased my Rails testing book, Everyday Rails Testing with RSpec, you can download an updated version from Leanpub that uses webdrivers.

This switch is a good move overall. webdrivers provides support for non-Chrome browsers, and is more actively maintained. And, from my experience, making the switch is mostly a one-line replacement. Open your Gemfile and make the swap:

 group :test do
-  gem 'selenium-webdriver'
-  gem 'chromedriver-helper'
+  gem 'webdrivers'
   # other test-only dependencies ...
 end

Since webdrivers brings in selenium-webdriver as a dependency, I've been removing it from my apps' own explicit dependencies. If you use Spring, you may need to stop it with bin/spring stop, then run your tests as usual. Assuming things were working before with chromedriver-helper, they should continue to work using webdrivers.

The only exception I've found to that is on a computer where I had a dev channel version of Chrome installed. While working to upgrade the source for my book, I ran into the following failure:

Failures:

  1) Tasks user toggles a task
     Failure/Error: visit root_path

     Net::HTTPServerException:
       404 "Not Found"
     # /Users/asumner/.rvm/gems/ruby-2.4.1/gems/webdrivers-3.7.1/lib/webdrivers/common.rb:109:in `get'
     # /Users/asumner/.rvm/gems/ruby-2.4.1/gems/webdrivers-3.7.1/lib/webdrivers/chromedriver.rb:26:in `latest_version'
     # /Users/asumner/.rvm/gems/ruby-2.4.1/gems/webdrivers-3.7.1/lib/webdrivers/common.rb:37:in `desired_version'
<... more stack trace ...>
     # ./spec/features/tasks_spec.rb:11:in `block (2 levels) in <top (required)>'
     # /Users/asumner/.rvm/gems/ruby-2.4.1/gems/spring-commands-rspec-1.0.4/lib/spring/commands/rspec.rb:18:in `call'
     # -e:1:in `<main>'

Finished in 4.71 seconds (files took 0.59138 seconds to load)
46 examples, 1 failure

Failed examples:

rspec ./spec/features/tasks_spec.rb:4 # Tasks user toggles a task

At first glance, I thought my app was returning a 404 for some reason, but that didn't make sense. After digging into the gem's source, I found the 404 was coming from Google's servers, attempting to download the latest ChromeDriver version from the dev channel. I was on a tight schedule, and replacing my dev channel Chrome with a standard install fixed the issue. (This may have been solved by now; I'll dig back in and update this post accordingly as I know more.)

I've also read about issues using webdrivers alongside VCR. I didn't run into these issues myself, but workarounds are now documented.

Thanks to Junichi Ito for sharing this issue with me, thanks to Mike Dalessio for his work on chromedriver-helper, and thanks to you for reading. I hope you find this information helpful.

7 reasons why learning SQL is still important for Rails developers

Aaron Sumner — Wed, 13 Mar 2019 01:57:15 +0000

Originally posted in Everyday Rails, but I don't think it only pertains to Rails developers! I'm curious to hear what folks who work in other languages and frameworks think.

I remember when I first learned SQL, back when I was writing Perl CGI scripts in a cave with my dinosaur friends. It was magical; a transformative step in my career as a software engineer. Without that initial understanding how to break business concepts down into discrete tables, then link and search across those tables with plain English (well, sort of) syntax, I wouldn't be where I am today.

Back then, my dinosaur coworkers and I had to programatically build out each INSERT, UPDATE, and SELECT ourselves. We didn't use an object relational mapper then (they may have existed; they just weren't on my team's radar). A lot of times, building out SQL queries was tedious and prone to error. So imagine my delight when I first picked up Rails and began replacing raw SQL with elegant Ruby code that performed the same database actions.

Today, a good 14 years after creating my first Rails app, I seldom have to write my own SQL. Active Record takes care of the tedium for me. It helps protect me from silly errors and security vulnerabilities, when used correctly. And, to be honest, my immediate recall of SQL has faded to the degree that I have to dust off some cobwebs and think a little harder for a moment about how to perform a task on those occasions Active Record isn't an option.

I recently got to thinking, with Active Record as robust as it is today, how important is it for a Rails developer to know any SQL?

While it's safe to say you can build an incredibly useful application with no knowledge of the language it's using to communicate with the underlying database, and leaning on Active Record is almost always preferable to handcrafted SQL in the context of a Rails application, having some degree of understanding of SQL comes in useful in at least seven scenarios.

Craft better queries with Active Record

What happens when you want to include data from a separate table in your query? If you know SQL, then you know that this is a job for a JOIN. Active Record conveniently includes a joins method to accomplish this, too, but if you don't know the lingo, or the concept behind it, you may find yourself searching blindly for an answer.

Perform queries that Active Record can't do

Active Record abstracts away a lot of SQL, but not all of it. For example, you couldn't do a LEFT OUTER JOIN using Active Record alone until Rails 5 was released. If your app is still on an older version, or you need to use a SQL function unique to your database vendor, then you'll need to write the SQL query by hand.

This comes with a downside: Part of the magic behind Active Record is the layer of abstraction it applies across multiple database vendors. Writing raw SQL in place of it can make your code less portable, meaning that if you ever choose to switch from one database engine to another, you may need to rewrite that query. Make sure you've got solid test coverage in place when taking this approach.

That said, I can only think of a couple of times when I've had to switch an application's database engine. So don't panic if you've got to write SQL by hand because Active Record doesn't support the query you need to perform directly.

Perform ad-hoc queries

Your app may have a robust administrative dashboard with beautiful reports presented in a slick user interface--but what do you do when asked about some bit of data that's not immediately available through the UI? Depending on your production setup, your only immediate access to the data may be through a database console. Knowing your way around the database through SQL alone can help answer such one-off questions promptly.

Improve application performance

Behind the scenes, Active Record has to perform a few extra steps to convert a query from a chain of Ruby methods to a SQL command to pass along to the database engine. Usually, it's fast enough. On occasion, though, it's not. In these cases, understanding how to compose a SQL query that gets the information you need quickly and accurately is critical.

The same caveat about database vendor lock-in applies here. If you use performance tricks available only to your current database vendor, you'll need to rewrite the query if you switch engines.

Understand legacy code

Active Record gains new features with each release of Rails. If your application has been around for awhile, you may run into queries that couldn't be performed by Active Record at the time, and thus are written by hand. With some understanding of SQL, you can understand what earlier developers needed to accomplish with the code and query in question. You can alter the query with confidence, and maybe even port it to a modern version of Active Record.

Of the reasons I've listed here, this is the one I run into most often in the codebases I work in. Raw SQL really stands out when juxtaposed against pretty Ruby code, but I know past developers didn't sneak it in there without reason. And sometimes, I'm able to rewrite the query using Active Record tools that weren't available to those past developers, and minimize context shifting for future developers.

Work outside of Rails

I've worked professionally with Ruby, Perl, Python, PHP, ASP, JavaScript, and probably a few other languages I'm forgetting, to develop backends for web applications. In all those scenarios, a SQL-based database was the constant. If you ever find yourself working with a different language or framework than Ruby on Rails, the techniques for connecting from the framework to the database may look radically different--but under the hood, there's a good chance it'll still be SQL. And understanding the fundamentals will help you pick up what's unique about said framework's approach to database interactivity.

Prepare for job interviews

This last one's maybe a little sassy, and definitely unsubstantiated--but it's also what got me thinking about the benefits of knowing SQL enough to write this article. As robust as Active Record is, job descriptions still often list (strong) understanding of SQL as a requirement. If you walked into an interview tomorrow morning, would you be able to demonstrate such understanding?

Now, I don't agree with the practice of grilling already nervous candidates questions about skills and concepts they'll seldom encounter on the job, but not everyone agrees with me on that. And as I've hopefully convinced you by now, there are plenty of other, legitimate reasons to keep some understanding of SQL in your back pocket, because you never know when it's going to be useful.