DEV Community: Ernesto Tagwerker (he/him)

Fortify Rails - Defending Your Ruby on Rails Applications from Bad Actors

Ernesto Tagwerker (he/him) — Mon, 26 Jun 2023 12:34:56 +0000

On Monday June 12th, 2023, FastRuby.io partnered with Expedited Security to talk about how to secure your Ruby on Rails application.

In this free webinar Ernesto Tagwerker (FastRuby.io) and Mike Buckbee (Expedited Security) discussed topics of interest related to Rails security (exploitable ActiveRecord code, vulnerable dependencies, botnets, DDoS, a breakdown of common threats, and more).

You’ll also get a sneak peak of Wafris, an Open Source service to prevent attackers and dark traffic to your application and of our new Rails Security Audit, a service to detect vulnerabilities and exploitable code in your app!

Check out the recording of the Fortify Rails webinar here: https://www.youtube.com/embed/KkXTw7UDR9w

Here are the slides on exploitable code and insecure dependencies: https://speakerdeck.com/etagwerker/fortify-rails-webinar

If you want to read more about Ruby and Rails security tooling check out this article: 4 Essential Security Tools To Level Up Your Rails Security

Need a trusted third party to give you a prioritized list of security issues? Check out our Rails Security Audit service.

Power Home Remodeling Increases Server Speed by 40% with FastRuby.io’s Tune Report

Ernesto Tagwerker (he/him) — Fri, 02 Jun 2023 10:10:59 +0000

Power Home Remodeling (Power) is the nation’s largest full-service exterior home remodeler and a top workplace.

Headquartered in Chester, Pennsylvania with offices in 18 territories across the United States, the award-winning company’s primary product line includes windows, siding, doors, roofing, solar roofing panels, and attic insulation.

In this article we will share how our Tune Report helped Power speed up their application by reducing their average page load time from 5 to 3 seconds.

Our Challenge

Power provides homeowners with top-rated customer service, and quality, energy-efficient products that help upgrade their home’s exterior. While a home remodeling company at its core, Power has also become a leader in workplace culture, leadership development, and technology.

Behind the scenes, Power’s unique, state-of-the-art collaborative intelligence platform—known as Nitro—enables seamless, real-time communication across the business.

The company’s technologists are responsible for developing and delivering Nitro’s software through continuous integration, as well as the three data centers that manage it.

According to Wade Winningham, Principal Developer, “We don’t use third-party applications for our sales, lead generation, tracking, and fulfillment. We’ve built everything ourselves and we’re proud that we operate our own platforms and technology without much reliance on an outside cloud.”

Wade, who helps set standards for the entire department, is always on the alert for ways to enhance and refresh their technology infrastructure. For the past six years, they have reached out to us anytime they needed support with Ruby on Rails maintenance, development, and upgrades.

Several years ago, the Ruby on Rails version underlying Power’s software was getting stale, but due to the size of their application, it was going to be a daunting task to handle internally.

We were hired to help with the upgrade from Rails 3.2 to 4.0. “OmbuLabs/FastRuby.io is an expert in Ruby and Rails, and we knew it was going to be in our best interest to hire them,” explained Wade.

“The Rails upgrade project was a huge success. The developers were top notch, knew what they were doing, understood our internal application, and helped us through it.” We collaborated with Power’s team to get them all the way to Rails 6.0 and then they continued their upgrade journey.

Our next challenge was to find ways for our clients to identify optimization opportunities within their Rails applications.

The Solution: Tune Report

Recently, in partnership with Nate Berkopec from Speedshop, we launched our new Tune Report, which helps our clients accelerate their Ruby applications, reach a higher level of performance, and reduce server costs.

We shared our new service with Wade and his team, and they immediately saw the benefits. “We thought it would be great to have an outside perspective of our application. The Tune Report would look at areas that we weren’t focusing on or even aware of having potential issues.”

See the results:

Our first step was to meet with everyone involved. The business technology team provided access to their repositories, internal hardware statistics, application metrics, and codebase to monitor and observe the application.

After we had a chance to explore, read, and analyze the source code, we delivered our Tune Report, which provided a list of key areas that could be immediately addressed and others that could be improved upon over time.

Doing More with Less: Improved Server Performance Results in Reduced Costs

Power’s applications run in a containerized environment. Their theory was that if they had more server instances and fewer processes running in those servers spread out over a wide area, that their application would be more resilient.

However, our Tune Report revealed an excessive amount of memory available, compared to the amount of CPU they were able to use within their servers.

Our recommendation was to reduce the number of server instances and increase the number of processes that each server executed.

For example, instead of having 60 servers with 2 processes, it would be far more efficient to have 20 servers with 6 to 8 processes each.

With more processes per server, there is less chance of one of those being busy when a request arrives. By making this adjustment, we were able to drive down the wait time from an average of 20-50 milliseconds to almost 0 milliseconds.

With this, and all other changes made so far from the report, page loads were reduced from 5 seconds to below 3 seconds.

“This was a huge win for us because we were able to size our application accordingly across multiple server instances,” said Wade. “It only took a few days to test this and verify that they were right.”

By running fewer server instances, hardware is freed up and computational power can be reallocated; ultimately, this results in cost savings and increased speed using half the server capacity.

We also found ways to improve query performance by adding indexes in the database. “These were simple for us to add and immediately improved performance across our application,” said Wade.

The Importance of Regular Checkups

For Wade, the Tune Report was like another set of eyes on areas they may not have thought to look at.

“It was great to have OmbuLabs/FastRuby.io’ insights. They came in, showed us where we could make improvements, and it was easy to implement their suggestions.”

Our team delivered a report that enabled Power to show noticeable results to non-technical stakeholders:

“Our executives were thrilled with the report,” shared Wade. “They liked the fact that we could take action immediately to experience an almost instantaneous impact, and recommended that we invest in a Tune Report on a regular basis.”

Wade felt the Tune Report was well worth the time and effort of getting an outside perspective and said:

“I was shocked that we were able to do so much in a short amount of time and to show that kind of return on investment. The Tune Report came at a really great time and showed us how beneficial it was to do—not just once, but routinely. It’s like taking your application to the auto mechanic for a checkup.”

“I really enjoy working with OmbuLabs/FastRuby.io. They’re experts at what they do, and I’ve always been impressed with their skill, professionalism, and knowledge,” concluded Wade. “They are also very involved in the Ruby community which just adds to their credibility.”

Summary

The collaboration between Power and OmbuLabs/FastRuby.io exemplifies how external experts can bolster in-house technological capabilities. The Tune Report proved an invaluable resource in optimizing Power’s Nitro platform, leading to improved performance and cost savings.

The success of this collaboration underscores the importance of regular performance audits and the value they bring in terms of enhancing user experience, reducing costs, and maintaining a competitive edge in a technology-driven world.

The Tune Report, product of a collaboration between FastRuby.io and Speedshop, proved instrumental in identifying optimization opportunities, leading to a significant decrease in average page load time, from 5 to 3 seconds.

One of the key recommendations, reducing the number of server instances and increasing the number of processes per server, resulted in a drop in wait time from 20-50 milliseconds to almost 0.

If you are interested in hearing more about our Ruby/Rails performance audits, send us a quick message.

WHO WE ARE:

OmbuLabs is Philadelphia's lean software boutique. Specializing in Ruby, Ruby/Rails Upgrades (FastRuby.io), JavaScript, JavaScript Upgrades (UpgradeJS.com), minimum viable product development, and reducing technical debt, we help companies (from startups to Fortune 500 companies) build and improve their digital products.

Founded in 2011 and headquartered in Philadelphia, Pennsylvania, our experienced and diverse team of developers is ready to do whatever it takes to help your business grow.

To learn more, visit us at OmbuLabs.com.

4 Essential Security Tools To Level Up Your Rails Security

Ernesto Tagwerker (he/him) — Wed, 31 May 2023 11:23:45 +0000

At FastRuby.io we love Ruby on Rails because it is so powerful: You can quickly create an application that is feature complete, stable, and secure

Unfortunately, maintaining a Rails application up to date and secure takes some effort.

In this blog post, we will cover a few Ruby gems and best practices that you can use to stay on top of your security, reliability, and stability needs.

Before we dive into 4 different Ruby gems you can use to improve your security support, let’s start with some of the security goodies that already come with Rails.

Rails’ Native Security Features

The good news is that Rails comes with a ton of security support, here are a few features that you are probably already using in your application:

Forcing SSL

You can easily secure your authentication by enabling TLS in your configuration:

# config/environments/production.rb
# Force all access to the app over SSL,
# use Strict-Transport-Security,
# and use secure cookies
config.force_ssl = true

If your web server (e.g. nginx) doesn’t force SSL, your Rails application will.

CSRF (Cross Site Request Forgery)

Some of these security features are enabled by default. For instance: CSRF. Ruby on Rails has specific, built-in support for CSRF tokens.

If you see this line in your application_controller.rb, you know that you are including Rails’ protection against cross site request forgery:

class ApplicationController < ActionController::Base
  protect_from_forgery
  # ...

CORS (Cross Origin Resource Sharing)

Occasionally, a need arises to share resources with another domain. For example, a file-upload function that sends data via an AJAX request to another domain.

In these cases, the same-origin rules followed by web browsers must be sent. Modern browsers, in compliance with HTML5 standards, will allow this to occur but in order to do this; some precaution must be taken.

For this, you can use the rack-cors gem:

gem 'rack-cors', :require => 'rack/cors'

You can configure it as a middleware like this:

# config/application.rb:

module Sample
  class Application < Rails::Application
    config.middleware.use Rack::Cors do
      allow do
        origins 'audit.fastruby.io'
        resource %r{/users/\d+.json},
        :headers => ['Origin', 'Accept', 'Content-Type'],
        :methods => [:post, :get]
      end
    end
  end
end

Filtering Parameters

To make sure sensitive request parameters aren’t logged to your log files, you can set up your application like this:

Rails.application.config.filter_parameters += [
  :credit_card_number,
  :password,
  :social_security_number
]

Values for these parameters will now show up as "[FILTERED]" in your log files.

Now that we have covered some of the key security features in Rails , let’s look at 4 gems that can help you level up your security best practices:

Bundler Audit
Brakeman
Rack::Attack
Secure Headers

Bundler Audit

This Ruby gem is quite useful for detecting versions of gems that are known to be vulnerable to security issues. bundler-audit uses an open database of vulnerable gems called ruby-advisory-db and compares it to the versions that show up in your Gemfile.lock.

You can install it and run it like this:

gem install bundler-audit
bundle-audit

If there are vulnerable versions in your Gemfile.lock it will show you something like this:

If everything looks good, it will show you something like this:

A few years ago we built a small, open source, web application to make it easier for us to share these vulnerabilities reports with our clients. You can find it and use it over here:

Bundler Audit Tool

It usually looks like this:

bundler-audit relies on ruby-advisory-db, a community-maintained database of known security vulnerabilities. While this is a great approach, it may also lead to gaps in coverage if a vulnerability is not yet documented in the database or if the database becomes outdated.

If you want to make sure that bundler-audit is running with the latest version of the ruby-advisory-db, you should always call it like this:

bundle-audit check --update

Brakeman

brakeman is another useful Ruby gem that is a static analysis security vulnerability scanner for Ruby on Rails applications.

You can quickly run it like this:

cd path/to/rails-app
gem install brakeman
brakeman

This will run a battery of “checks” against your source code and report any potential security issues:

Loading scanner...
Processing application in /Users/etagwerker/Projects/ombulabs/foobar
Processing gems...
[Notice] Detected Rails 6 application
Processing configuration...
[Notice] Escaping HTML by default
Parsing files...
Detecting file types...
Processing initializers...
Processing libs...
Processing routes...
Processing templates...
Processing data flow in templates...
Processing models...
Processing controllers...
Processing data flow in controllers...
Indexing call sites...
Running checks in parallel...
 - CheckBasicAuth
 ...
 - CheckYAMLParsing
Checks finished, collecting results...
Generating report...

== Brakeman Report ==

Application Path: /Users/etagwerker/Projects/ombulabs/foobar
Rails Version: 6.1.4
Brakeman Version: 5.4.1
Scan Date: 2023-05-02 15:18:55 -0400
Duration: 0.430032 seconds
Checks Run: BasicAuth, ..., YAMLParsing

== Overview ==

Controllers: 3
Models: 2
Templates: 15
Errors: 0
Security Warnings: 2

== Warning Types ==

Cross-Site Scripting: 1
Unmaintained Dependency: 1

== Warnings ==

Confidence: High
Category: Unmaintained Dependency
Check: EOLRuby
Message: Support for Ruby 2.7.2 ended on 2023-03-31
File: .ruby-version
Line: 1

Confidence: Weak
Category: Cross-Site Scripting
Check: SanitizeConfigCve
Message: rails-html-sanitizer 1.3.0 is vulnerable to cross-site scripting when `select` and `style` tags are allowed (CVE-2022-32209). Upgrade to 1.4.3 or newer
File: Gemfile.lock
Line: 194

To see a complete list of checks ran by Brakeman, you can find them over here: List of Brakeman Checks

This sample application is in pretty good shape, but it does report two warnings:

High: I’m using Rails 6.1 with Ruby 2.7.2. Security support for Ruby 2.7.x ended on March 31st, 2023.
Weak: I’m using a version of rails-html-sanitizer that is known to be vulnerable.

One of them is quite easy to fix: bundle update rails-html-sanitizer and the warning goes away. We can safely deploy this to production with a small pull request.

The other warning might be harder to accomplish: We need to upgrade from Ruby 2.7 to 3.0.

Just like any other static analysis tool, brakeman may generate false positives or negatives. This means that it might report issues that aren’t actually problems or fail to detect actual vulnerabilities. We need to be cautious when interpreting the results and not rely solely on these tools for ensuring security.

Rack::Attack

rack-attack is a middleware that can be used in your Rails application (and any rack-based Ruby applications!) to protect it from bad clients.

Rack::Attack lets you easily decide when to allow, block and throttle based on properties of the request.

Here are some of its key features:

Safelisting / Blocklisting

You can quickly allow an IP like this:

# config/initializers/rack_attack.rb (for rails app)
Rack::Attack.safelist_ip("5.6.7.0/24")

This could be useful if you want to allowlist your development team’s IP addresses.

You could quickly block an IP like this:

# config/initializers/rack_attack.rb (for rails apps)
Rack::Attack.blocklist_ip("1.2.3.4")

It’s a powerful combination to combine rack-attack with environment variables like this:

# config/initializers/rack-attack.rb

bad_ips = ENV["BLOCKLIST_IPS"].split(",") # to be blocked

Rack::Attack.blocklist "blocklist bad IP address" do |req|
  bad_ips.include?(req.ip)
end

good_ips = ENV["SAFELIST_IPS"].split(",") # to be safelisted

Rack::Attack.safelist "safelisted good IP address" do |req|
  good_ips.include?(req.ip)
end

Throttling

This would be a good way to slow down an attacker who is trying to sign in to a user account using a brute force attack:

# Throttle login attempts for a given email parameter to 6 reqs/minute
# Return the *normalized* email as a discriminator on POST /login requests
Rack::Attack.throttle('limit logins per email', limit: 6, period: 60) do |req|
  if req.path == '/login' && req.post?
    # Normalize the email, using the same logic as your authentication process, to
    # protect against rate limit bypasses.
    req.params['email'].to_s.downcase.gsub(/\s+/, "")
  end
end

Additionally, you could have a more broad criteria for limiting the number of requests per minute:

# config/initializers/rack_attack.rb (for rails apps)

Rack::Attack.throttle("requests by ip", limit: 5, period: 2) do |request|
  request.ip
end

Based on the IP of the user, this configuration would limit their rate to 5 requests every 2 seconds.

You can combine rack-attack and Cloudflare as a good way to prevent DDoS attacks. Just because you are using rack-attack, it doesn’t mean that you should not use a tool like Cloudflare.

Secure Headers

The secure_headers gem will automatically apply several headers that are related to security. This includes:

Content Security Policy (CSP) - Helps detect/prevent XSS, mixed-content, and other classes of attack. Default value: default-src 'self' https:; font-src 'self' https: data:; img-src 'self' https: data:; object-src 'none'; script-src https:; style-src 'self' https: 'unsafe-inline'
HTTP Strict Transport Security (HSTS) - Protects from SSLStrip/Firesheep attacks. Default value: max-age=631138519
X-Frame-Options (XFO) - Prevents your content from being framed and potentially clickjacked. Default value: sameorigin
X-XSS-Protection - Cross site scripting heuristic filter for IE/Chrome. Default value: 1; mode=block
X-Content-Type-Options - Prevent content type sniffing. Default value: nosniff
X-Download-Options - Prevent file downloads opening. Default value: noopen
X-Permitted-Cross-Domain-Policies - Restrict Adobe Flash Player’s access to data. Default value: none

While secure_headers provides sensible safe default settings, it is essential to tailor them to your application’s specific needs.

Additional Resources

If you’re interested in reading more about securing your Rails applications, here is a list of useful resources:

Conclusion

While Rails provides a lot of security features, there are at least 4 ways to level up your application’s security with these great tools:

bundler-audit to find dependencies that are known to have vulnerabilities
brakeman to find idioms/calls that could be dangerous for your application
rack-attack to defend your application against bad, abusive clients
secure_headers to quickly apply several security headers to all your responses

This list is by no means exhaustive but it shows some of the key tools you can use to stay on top of potential security issues.

Although tools like bundler-audit and brakeman are helpful in identifying potential vulnerabilities, they should not be solely relied upon.

As a team, you should define a comprehensive security strategy which includes regular code audits, code reviews, and penetration testing.

Happy hacking!

PS: Need a code security audit? Contact us!

Introduction to Rails Engines

Ernesto Tagwerker (he/him) — Tue, 23 May 2023 18:58:06 +0000

Rails Engines are an architectural pattern that can be used to modularize a Rails application. They are self-contained applications that can be mounted within a larger Rails application. In this post, we will dive into the world of Rails Engines and explore what they are, how to create them, how to use them, when to use them, and why they are important.

What are Rails Engines?

Rails Engines are essentially mini-applications that can be plugged into a larger Rails application. They allow you to modularize your code and keep it separate from the core application. This makes it easier to maintain and update the codebase as a whole.

Engines can include models, controllers, views, and assets, and they can also have their own routes and configurations. For example Devise, an engine that provides authentication for a host application, has its own views, controllers, routes and configurations.

It is worth mentioning that Rails Engines are similar to Railities.

However, they differ in that Railities are simple Ruby modules that includes various hooks into the Rails initialization process, such as configuring middleware or adding new generators, whereas a Rails Engine has a more complex directory structure and includes everything needed to function as a standalone application, such as controllers, models, and views.

How to Create a Rails Engine

Creating a Rails engine is a relatively simple process. To create a new engine, run the following command in the terminal:

rails plugin new payments --mountable

This will create a new Rails engine in the payments directory with the --mountable option, which means that the engine can be mounted within another Rails application. Please note that the --mountable option is very important as running the above command without it will create a typical plugin which is not mountable. Below are the files generated by the above command:

$ bin/rails plugin new payments --mountable
      create  
      create README.md
      create Rakefile
      create payments.gemspec
      create .gitignore
      create Gemfile
         run git init from "."
Initialized empty Git repository in /ombulabs.com/payments/.git/
      create app
      create app/controllers/payments/application_controller.rb
      create app/helpers/payments/application_helper.rb
      create app/jobs/payments/application_job.rb
      create app/mailers/payments/application_mailer.rb
      create app/models/payments/application_record.rb
      create app/views/layouts/payments/application.html.erb
      create app/assets/images/payments
      create app/assets/images/payments/.keep
      create app/models/concerns
      create app/models/concerns/.keep
      create app/controllers/concerns
      create app/controllers/concerns/.keep
      create config/routes.rb
      create lib/payments.rb
      create lib/tasks/payments_tasks.rake
      create lib/payments/version.rb
      create lib/payments/engine.rb
      create app/assets/config/payments_manifest.js
      create app/assets/stylesheets/payments/application.css
      create bin/rails
      create test/test_helper.rb
      create test/payments_test.rb
      create test/fixtures/files
      create test/fixtures/files/.keep
      create test/controllers
      create test/controllers/.keep
      create test/mailers
      create test/mailers/.keep
      create test/models
      create test/models/.keep
      create test/integration
      create test/integration/.keep
      create test/helpers
      create test/helpers/.keep
      create test/integration/navigation_test.rb
  vendor_app test/dummy
      append /ombulabs.com/Gemfile

Once you have created your engine, you can add models, controllers, views, and assets just like you would in a regular Rails application. You can also define your own routes and configurations within the engine.

How to Use Rails Engines

Using a Rails Engine is also straightforward. The engine needs to be specified inside the Gemfile and run bundle install:

gem 'payments', path: 'payments'

To make our Payments engine’s functionality accessible from within our application, we need to mount it in the host application’s config/routes.rb file. However, it is not always the case as we can use the engine functionality without mounting it:

mount Payments::Engine, at: "/payments"

This will mount our Payments engine at the specified path within your application. You can now use the models, controllers, views, and assets from the engine just like you would in a regular Rails application.

When to Use Rails Engines

Rails Engines are particularly useful when you want to modularize your codebase and keep it separate from the core application. This is especially helpful when you are working on a large project with multiple developers, as it allows you to separate responsibilities and avoid conflicts.

Engines can also be used to create reusable components that can be shared across multiple applications. For example, in an e-commerce context, you could use Rails Engines to build separate modules for different aspects of the site, such as product catalog, shopping cart, payment gateway, and order management.

Each module could be developed and tested independently and then mounted within the main Rails application. This approach would allow you to scale each module independently as needed, and also make it easier to reuse the code across different e-commerce sites.

Why Rails Engines are Important

Rails Engines are important because they provide a modular way to organize and share code within a larger Rails application. Here are a few key benefits of using Rails engines:

Code Reuse: Rails engines allow you to extract reusable code into separate modules, making it easier to share code across multiple projects. This can save time and effort in development, as you can develop and test functionality once and then reuse it in different contexts.
Modularity: By breaking up your Rails application into smaller, self-contained modules, you can make your codebase more manageable and easier to maintain. Each engine can be developed and tested independently, allowing you to focus on specific areas of functionality without worrying about breaking changes in the rest of the application.
Encapsulation: Engines provide a way to encapsulate functionality within a well-defined interface. This makes it easier to reason about the code and reduces the risk of introducing bugs or breaking changes in other parts of the application.
Scalability: Engines can be used to scale your application horizontally by allowing you to add or remove functionality as needed. This can help you to adapt your application to changing business requirements and user needs without having to make major changes to the overall architecture.
Customization: Rails engines can be customized to meet the specific needs of different projects or clients. This can be particularly useful in agency or consultancy contexts, where you may need to develop similar functionality across multiple projects but with slight variations in requirements.

By keeping your code organized and modularized, you can improve the overall quality and maintainability of your application.

Conclusion

In conclusion, Rails engines provide a powerful way to organize and share code within a larger Rails application. By breaking up your application into smaller, self-contained modules, you can make your codebase more manageable and easier to maintain. Engines allow you to extract reusable code, encapsulate functionality, and scale your application horizontally by adding or removing functionality as needed. Additionally, engines can be customized to meet the specific needs of different projects or clients.

While Rails engines may not be appropriate for every project or context, they can be a valuable tool for developers looking to create more modular, maintainable, and scalable applications. Whether you’re building an HR management system, an e-commerce site, or a content management system, Rails engines can help you to develop and reuse functionality in a more efficient and effective way.

For further reading you can go through the Rails guide for Engines.

Need help modularising your Rails application? Contact us and have the FastRuby.io experts help you with it!

Fixed-cost, Monthly Maintenance Services

Ernesto Tagwerker (he/him) — Tue, 23 May 2023 10:23:45 +0000

Ever since we started offering productized Ruby and Rails upgrade services and upgrade roadmaps, we’ve been interested in helping as many people and companies as possible.

Unfortunately, in the past we’ve had to turn down companies who wanted to work with us but couldn’t secure the minimum monthly budget to work with our experts.

I’m pleased to announce that we’re now offering new opportunities for startups and small businesses to work alongside our team.

In this article, I will share a few new options to collaborate with our team of experts who specialize in technical debt remediation.

Why

Recently I had the opportunity to connect with a lot of SaaS founders at MicroConf ‘23 which made me realize that there are many entrepreneurs, startups, and small businesses out there that want to pay off technical debt by investing a small amount of money per month.

While teams understand the need to remediate their technical debt, this very often loses priority to shipping features, releasing patches, and addressing other tasks prioritized in their product roadmaps.

Although everybody would like to upgrade their app as fast as possible, sometimes that’s just not in the budget. Instead, a good alternative can be to gradually work through your technical debt with a slower-paced retainer model.

What's included in our monthly maintenance services

Our fixed-cost, monthly retainer will include services to gradually reduce the technical debt in your application. Our services will include:

Ruby Upgrades
Rails Upgrades
Dependency Management
Performance Monitoring
Tech Debt Management
Security Patches

Ruby Upgrades

If your application could use a Ruby upgrade, we will submit a pull request that looks like this:

(Source: Upgrade to Ruby 3.2 on Points)

If your application is not properly configured to dual boot with next_rails, we will submit a PR that looks like this:

(Source: Set up dual booting on Benchmark.fyi)

This will be the basis of our work around dependency management.

We will make sure that your application can run with both versions of Ruby. If you are on Ruby 2.7, we will make sure that your test suite works with both Ruby 2.7 and 3.0.

Once we fix all the errors in your test suite, we will make sure that your staging environment works with the next version of Ruby. If anything goes wrong, we will make sure we patch all issues before we deploy the change to production.

Rails Upgrades

Similarly, we will gradually tackle Rails upgrade projects. We will use our years of experience to submit many, tiny pull requests that will upgrade your application taking baby steps.

We will make sure that your application can run with both versions of Rails. If you are on Rails 5.2, we will make sure that your test suite works with both Rails 5.2 and 6.0.

Once we fix all the errors in your test suite, we will make sure that your staging environment works with the next version of Rails. If anything goes wrong, we will make sure we patch all issues before we deploy the change to production.

If you are curious about the dual booting technique, we have been running the Rails upgrade workshop at RailsConf since 2019. Here is a recording of the workshop we delivered at RailsConf 2021

Dependency Management

While there are great tools out there that can quickly help you upgrade minor versions of your dependencies (e.g. Depfu), there seems to be a missing link between what we do (Ruby/Rails Upgrades) and what they do (shipping automated pull requests, upgrading one minor version at a time).

That’s where we come in. We will work on minor and major version upgrades (according to semver) for all of your dependencies.

We will submit one PR at a time. This will give us the opportunity to properly test, deploy, and monitor any issues in production.

Performance Monitoring

If your application is not currently using an Application Performance Monitoring (APM) service (e.g. New Relic), we will set one up for you. We’ll consider your needs and budget, to make the best recommendation.

Once your APM is up and running, we will keep track of low performance and high memory load issues. We will share a monthly report with the top issues we noticed as Ruby and Rails performance experts.

If you want a more in-depth analysis of your application’s performance, we can deliver our highly-rate and always insightful Tune Report.

While the Tune Report is not included in any of the monthly plans, our clients have found that they can recover their investment in about 3 months.

Tech Debt Management

As open source maintainers of technical debt projects like RubyCritic and Skunk we know how important it is to keep track of your code complexity, churn, and code coverage metrics.

On top of that, we know how difficult it is to communicate the problems with consistently increasing technical debt:

Reduced development velocity
Constant regressions
Undesired side effects when deploying small changes

We can help with that! Every month you will get a report on your technical debt, not just on the codebase, but also in your development workflow, infrastructure, and best practices.

In this report we will include any changes that were made to simplify complex code structures, enhance your development workflow, or anything else that was shipped to make your codebase easier to maintain.

Our goal is to gradually improve code quality without disrupting your development team’s operations.

If you don’t have a shared understanding of what “code quality” really is, we can help you define one that works for your team. After that we can help you “codify” your code quality definition with tools like reek, rubocop, andrubycritic.

Security Patches

Picture this: On day one of our engagement our team will perform a quick security assessment of your codebase. This will give us an idea of the work to be done to secure your application from known exploits and potential attack vectors.

Every month we will make security patches that will level up the security standards for your Rails application. Depending on the severity of the issues we find, we might prioritize this topic on day one.

Pricing

Are you ready to make your application more maintainable?

Considering our 10+ years of experience upgrading applications, and more than 20,000 developer hours paying off technical debt, we believe we can offer meaningful updates to your codebase starting at $1,000 per month:

Shito : $1,000 per month
Shohin : $2,000 per month
Komono : $3,000 per month

Each package includes a maximum number of hours per month that we can invest in your application. When naming these packages, we decided to go with Bonsai types because we see a lot of similarities between pruning Bonsai trees and gradually paying off technical debt. 🌳

Shito

This is a great package for organizations and companies that either don’t have a lot of technical debt or want to go slow.

Their test suite might be well-written, their Rails application might be a small monolith, or their development team would be overwhelmed with more than 2 pull requests per month.

Shohin

This is a great package for organizations and companies that have some technical debt, know they need to fix it, and are ready to review from 2 to 4 pull requests per month.

Their test suite might need some attention, their Rails application might have between 50 and 150 models, or they’re one or two versions behind the latest stable release of Ruby and Rails.

Komono

This is a great package for organizations and companies that are ready to ship 3x more than if they had chosen the Shito package.

Usually their Rails application is rather large, their main dependencies (Ruby and Rails) are a few versions behind, their test suite is flaky, or they need to move fast and upgrade things.

Depending on the plan that you pick, we will invest from 5 to 20 hours per month remediating technical debt in your application/s.

Strategic Projects

We know that every now and then our clients will have one-off, strategic projects they will want to ship as soon as possible and we want to be able to help.

When that happens, we can offer our services on an hourly basis to ship value in the form of features, infrastructure upgrades, bug fixes, or anything that will make a difference in our clients’ businesses.

Rescue Services

Our monthly packages don’t include rescue services. If your application needs to be rescued, we can certainly help! We’ve done this many times in the past and we are happy to get you out of a thorny situation.

For these projects, we need to start with a short, one-week retainer to kickstart the rescue project. This will give us time to assess the situation, come up with a rescue plan, and start executing it.

How does it work?

We are used to working with teams of all sizes. From engineering teams of one to teams of 50+ software engineers, we see every engagement as a successful collaboration between our team and your team.

We thrive when we collaborate with our clients’ engineering team on a daily basis, so we will have a channel to communicate asynchronously. If you need to get on a call, we can do that too!

Any of the changes that we submit will be open for review and discussion. We will clearly explain why they are necessary, what the changes will do, and what impact they will have in your tech debt scorecard.

Communication is key to us! We provide you with a monthly report on your technical debt, including codebase, development workflow, infrastructure, and best practices. We are always open to hearing your thoughts and needs and answering your questions.

Summary

Our commitment to helping people and businesses overcome their technical debt is stronger than ever.

We are excited to launch our new monthly maintenance packages tailored for non-profit organizations, startups and small businesses. We are certain we will gradually reduce technical debt in their applications.

Our technical debt remediation services, provided by a team of experts, include dependency management, Ruby upgrades, Rails upgrades, performance monitoring, tech debt management, and security patches.

These three affordable plans starting at $1,000 per month are cashflow-friendly, consistently add value to your codebase, and give you access to our team of legacy code experts.

You can expect to see gradual improvements in your code quality and overall decreasing technical debt while keeping your development team’s velocity uninterrupted.

“The typical development organization can increase their feature delivery efficiency by at least 25% by managing technical debt. That’s the equivalent of having 25% more developers without additional staffing costs or** coordination needs**.”

(Source: CodeScene Whitepaper)

What would you do if you had 25% more development efficiency? How much more could your development team deliver in a month?

Let us be your trusted partner in making your application more maintainable, secure, and efficient.

Ready to pay off technical debt? Send us a message!

7 Common Mistakes in Rails Upgrades

Ernesto Tagwerker (he/him) — Tue, 25 Apr 2023 00:30:00 +0000

(This article was written by Rishi Jain and originally published in the FastRuby.io Blog)

Ruby on Rails is a popular web application framework that is constantly evolving with new versions being released frequently. While upgrading to a newer Rails version can bring new features, better performance, and security patches/improvements, it can also be a challenging task.

In this blog post, we will discuss 7 common mistakes made while doing Rails upgradesand how to avoid them.

Skipping Versions

We have seen customers trying to upgrade Rails from 5.2 to 7.0 and Ruby from 2.5 to 3.1 all under one single pull request and failing to deploy a successful version on production.

The problem with this approach is that it becomes extremely difficult to point out where the problem lies when your user encounters a bug in the app.

Did it happen when you upgraded from Rails 5.2 to 6.0 or in any other jump? Did it happen due to a deprecation warning that was supposed to be handled in Rails 6.0 to 6.1 jump but you skipped that as you tried to upgrade directly from Rails 5.2 to 7.0?

Another problem here is that you miss the chance to achieve small wins by deploying relatively small upgrades to production and thus exposing yourself to greater chances of encountering bugs and downtime to your app.

The only way to avoid these problems is to take a step-wise approach of upgrading only one major or one minor version at a time. If we were to upgrade an app from Rails 5.2 to 7.0, we would first upgrade it to Rails 6.0, deploy to production, then upgrade to 6.1, deploy to production, and then upgrade to 7.0 and deploy to production.

The Rails Guides also have documentation for changes involving each version jump which makes your job easier. For example: here is a documentation guide for upgrading the app from Rails 6.1 to 7.0.

If you also have to upgrade Ruby along with Rails, then the ideal approach is to upgrade them individually. This way you can minimize the risk of bugs creeping in.

Ignoring Deprecation Warnings

We have seen some clients silencing the deprecation warnings instead of addressing them. The reason deprecation warning exists is to alert developers of a behavior of the framework, ruby, or another gem that will not be supported in the future and to give them time to move away from the deprecated behavior which will eventually not be supported.

The problem with silencing the warning is when the deprecated behavior is no longer supported, it will break the application and the developer will have to either roll back the upgrade or make changes to the app really quickly and no developer would like to be in that situation.

Additionally, addressing the deprecation warnings are fairly straightforward most of the time, and the deprecation warning message also indicates how to fix / change the code to get rid of the warning.

It is way easier to address a deprecation warning that is explicit (it tells you the code, the change and the line that’s triggering it) than fixing a bug that you don’t know where it’s happening and what is causing it.

We’ve seen hours of work of debugging to end up finding that the bug was due to something that had a deprecation warning which was never addressed.

Upgrading Gems

When you are upgrading Rails, chances are that you are going to upgrade other gems along with it. But, how do you find out which gems need to be upgraded and to what versions? There are 2 ways to go about this.

First, running bundle update and then it will update every gem to the most compatible version. The problem with this approach is that since it is going to_update all the possible gems_ with a compatible higher version, you are exposing yourself to more bugs which can come due to code / behavior changes in the upgraded gems.

Second, running bundle update rails and then it will break with incompatibility messages across your gem set and with errors indicating which gems need to be updated. The downside to this approach is that it takes longer as you have to manually upgrade gems iteratively until the bundle update is successful.

The upsides are huge though as chances of bugs creeping comes down as you have only upgraded gems that absolutely needed to be upgraded.

You can read more about it here.

Undocumented Monkey Patches and Forking gems

Yes, Ruby is great because the powers it gives you and one of them is monkey patching existing behavior. This is great because it solves your problems in the short term.

But we have seen enough projects where the monkey patched behavior is left with no comments explaining why it was done in the first place and the developer who did this is no more working at the organization to explain and now you are left with no option but to make sense of the patch work all by yourself which is very close to guess work.

Another big problem we have encountered over the years is forking gems. Lots of times, developers fork open source gems and make changes to it to fit their needs.

But those changes are never submitted to the open source project and the reason for the fork is left to yourself without documenting it. Now it is the team who is working on the upgrade that is left to make sense of why the fork was done, and is it safe to go the released rubygems version of it or make the fork work with the upgraded Rails and other gems.

Lack of test coverage or testing strategy

To ensure that the upgrade of the Rails app is successfully released on production, you need either of the 2 things at least.

First, a good test coverage of your application. Anything above 80% test coverage is considered good if it includes coverage of the critical paths of the applications.

Second, awareness of all the critical paths of the application and ability to test them manually.

If you do not have either of the 2 things in place, it increases the chances of encountering bugs upon releasing it on production. Even with both the strategies in place, you can still set yourself for a buggy release on production and that can be due to not having a dedicated QA team or QA role defined in your workflow.

When the developer doing the code review on the pull request ends up doing the QA of it as well it is not an ideal situation. It can lead to bugs not caught before they are released to your users.

Not using the right tools

There are many ways to go about upgrading your application, but choosing the right tools and the right strategy can go a long way in making your experience smooth.

Something as trivial as branching and code arrangement strategy holds a lot of importance in your workflow. We have seen a lot of teams struggle or end up doing repetitive work because of lack of clearly defined strategy.

To avoid confusion on how to run the upgrades, we contribute and maintain the next_rails gem which:

Helps us in running 2 different versions of Ruby or Rails in the same branch
Facilitates running specs for the 2 different versions of Ruby or Rails from the same branch

This goes a long way in debugging issues while doing upgrades without switching branches. It also helps in implementing a gradual deployment strategy with Kubernetes.

We have also seen customers trying to do everything “the Rails way” when it does not make sense for them.

For example, Active Storage is a great gem and very useful for someone allowing users to upload files. But one of our customers wanted to use Active Storage even though their use case did not fit to use it.

What they already had made more sense for them. Using Active Storage would have made their systems more complex. So it is also important to know when to move away from the defaults.

Not Having a Rollback Plan

Finally, not having a rollback plan is a common mistake made while upgrading Ruby or Rails.

Despite careful planning, unexpected issues can arise during the deployment. It is important to have a rollback plan in case of any issues that may arise during the deployment process.

A rollback plan should include steps to revert to the previous version of Rails, restore db backups, and fix issues that caused the rollback.

Conclusion

Upgrading your Ruby or Rails version is essential for keeping your application secure and up to date.

However, upgrading can be a challenging task, especially when you make common mistakes.

In this article, we discussed 7 common mistakes that developers make when upgrading Ruby or Rails and how to avoid them. By avoiding these mistakes, you can make the upgrade process smoother and less stressful.

In conclusion, upgrading Rails can be a daunting task, but with the right approach, you can make it a seamless process.

Happy upgrading!

PS: Need help upgrading Ruby or Rails to their latest stable version? We have some availability to help your team upgrade Ruby/Rails!

How to Upgrade Rails Without a Test Suite

Ernesto Tagwerker (he/him) — Wed, 13 Jan 2021 09:00:00 +0000

Sometimes you will be caught between a rock and a hard place. For example: You really need to upgrade your Rails application because Heroku is discontinuing your stack in two weeks so you don't have the time to write an entire test suite.

Sometimes the situation will be that your boss really needs you to upgrade Ruby or Rails to use a version that gets security updates and they won't allow you to write tests beforehand.

This article will explain how to ship a Rails upgrade project without a test suite. If your application has no tests, you will learn how to upgrade without having to write tests first.

Set up Staging
Check the Changelog
Write a Testing Script
Set up Error Tracking
Set up Dual Booting
Upgrade Development and Test
Deploy to Staging and Test
Deploy to Production
If you have the time, write integration tests
Summary

1. Set up Staging

Have a staging server that is identical to your production server. You will need to thoroughly test your changes in a staging environment before you deploy any changes to production.

Strive to make this environment as similar to production as possible. Fill it up with production-like data, so that you can test real world scenarios.

2. Changelog

Many of the changes in the new Rails version won't affect your application. Read the most notable changes in the changelog and manually find whether the APIs that changed are getting used in your application's code.

In this step there will be a lot of grepping or finding in files within your code editor. If you want to learn how to do this, we wrote an article about that: How we upgrade Rails applications

3. Write a Testing Script

If you know the application from top to bottom, write down several manual test scenarios. If you don't, work with the product owner to define a set of steps that you can run to smoke test the application in your development, staging, and production environments.

4. Set up Error Tracking

If you don't have this configured, make sure that you set it up before you deploy anything to staging or production.

There are many options when it comes to error tracking. Some of the most popular alternatives are Sentry; Honeybadger; or Rollbar. This will come in handy when you deploy your code and you get real users using your application on the new version of Rails.

Considering your application does not have a test suite, I can guarantee that you will get exception notifications once your code is live. You want to make sure that you can get all the necessary information so that you can quickly patch those bugs.

Even applications with a solid test suite and great code coverage will find exceptions in production. So this is a great idea even if you are not deploying a major Rails upgrade. :)

5. Set up Dual Booting

This will help you with the next steps across all your environments. Using this technique you will save a lot of time, because you will be able to run your application with Rails X or Y by changing a single environment variable.

We have explained how to dual boot in this article:How to Dual Boot Ruby on Rails

6. Upgrade Development and Test

If you have set up dual booting correctly, you will be able to run your current Rails version like this:

bundle exec rails server

And your new Rails version like this:

next bundle exec rails server

Now you can start smoke testing your application in your development environment. This step will include a lot of steps that look like this:

Read and execute your testing script
Monitor your logs and error tracking service
Find issues
Patch them
And repeat...

Once you feel comfortable that you have tested your critical user paths and everything looks good, you can move on to deploying your changes to your staging environment.

7. Deploy to Staging and Test

Now you are at a point where you can involve your QA team. Once you have deployed and tested your changes in staging, you can involve more people in helping you manually test critical user flows.

8. Deploy to Production and Test

There are two safe-ish ways to deploy your Rails upgrade to production:

Gradual deployment
Full deployment

Gradual deployment

This is another step that benefits from your dual booting code. You can deploy your changes to only a few application instances in your cluster. Then you will have your cluster running both versions in parallel (Rails X and Y).

You can set up your production environment to direct only 10% of the traffic to the new version of your code. This article explains how you can do that:Continuously Integrating a Major Rails Upgrade

Full deployment

If your application does not get a lot of traffic at night, you might want to deploy your changes to all your visitors then. This approach is certainly riskier because slower traffic means fewer users which means fewer "testers" for real world scenarios.

In both scenarios, you will eventually sit and wait for errors. Depending on the severity of the errors, you might want to roll back your changes and go back tostep 6.

If there are only a few minor bugs, you might want to hot fix them in production.

9. Integration Tests

If you have the time, I would highly recommend writing some integration tests. You can use RSpec and Capybara to write some simple tests to view them in a browser.

You don't have to get too creative when writing these tests, you can basically write something like this:

# spec/features/homepage_spec.rb
require "rails_helper"

RSpec.describe "Homepage at FastRuby.io", js: true do
  it "does not blow up" do
    visit "/"

    expect(page).to have_text("Need to upgrade Rails but don't have the time?")
  end
end

You can write basic, smoke-testy integration tests that mimic the behavior of end users and get to a non-zero amount of code coverage in a few minutes. If you have the time, doing this will save you time in your workflow.

Summary (TL;DR)

Someone is going to test your application. It could be you, writing automated tests and increasing code coverage. It might be your QA team, testing things work as expected in staging and production. In the worst case, it will be your end users trying to do something in production.

If you don't have a test suite, it does not mean that you cannot upgrade a dependency of your application. It usually means that it will take longer than if you had a well-written test suite, because there will be a lot of manual testing during the project.

How to Prepare Your Rails App for an Upgrade

Ernesto Tagwerker (he/him) — Tue, 29 Dec 2020 09:00:00 +0000

This article is part of our Rails Upgrade series. To see more of them, click here.

This article will cover the most important aspects that you need to know to prepare your Ruby on Rails application for working on an upgrade.

Test Coverage
Staging and Production
Patch Version
Incompatibilities
Dual Boot

1. Code Coverage

Not all applications are good candidates for a Rails upgrade project. We strongly advise against upgrading big applications which are running in production with minimal test coverage.

Make sure that you have at least 80% test coverage before starting the upgrade project. If you don't have a solid test suite (or a dedicated QA team), you will likely find many problems which will force you to roll back the upgrades as soon as they hit production.

You can easily calculate your coverage by using simplecov. Here is a tutorial about how to use it.

2. Staging and Production

We advise all of our clients to follow a Git flow workflow and to actively manage at least two environments: staging and production.

Every change should be code reviewed using pull requests and it should run the entire test suite. Once all checks have passed, changes should be deployed to staging. After your QA team has tested your critical flow in staging, you can deploy the changes to production.

3. Patch Version

Before working on an upgrade you should make sure that you are running the application with the latest patch version. That way, you make sure that the latest security patches have been installed. On top of that, with more recent versions, you will get a series of deprecation warnings that will be useful in finding out what needs to be done.

4. Incompatibilities

Have you ever gotten tangled in dependencies? You need to upgrade dependency A, but that's incompatible with Rails 5.0 because dependency B (which is a dependency of A) is not compatible with that version of ActiveRecord).

Before you start going down the rabbit hole, make sure to check your Gemfile.lock for incompatibilities. For that, you can use this website: https://railsbump.org/

That will give you an idea of how many dependencies are not compatible with Rails 4, 5, or 6.

5. Dual Boot

To help you switch between your current Rails version and the new one, you can create a dual boot mechanism. The fastest way is to install the handy gem next_rails. You can initialize it by doing

$ gem install next_rails
$ next --init

You can then set up two Rails versions in your Gemfile like this:

if next?
  gem 'rails', '~> 6.0.0'
else
  gem 'rails', '~> 5.2.3'
end

Sometimes dependencies are backwards compatible with the current version of Rails. Within the libraries, you will find code that looks like this:

if Rails::VERSION >= 5.1
  # does X
else
  # does Y
end

If that is the case, then you might be able to just upgrade the dependency using bundle update.

Next Steps

Now that you have laid the groundwork for you upgrade, it is time to start the work itself. For details on what you specific version upgrade will require, check out the corresponding guide:

Churn vs. Complexity vs. Code Coverage

Ernesto Tagwerker (he/him) — Wed, 13 May 2020 02:25:00 +0000

Churn vs. Complexity analysis is a great way to find insights about the maintainability of a project. Two of my favorite authors have written great articles about the Churn vs. Complexity graph:

Getting Empirical about Refactoring by Michael Feathers
Breaking up the Beheamoth by Sandi Metz

This two-dimensional graph can be very useful in finding the files that are the hardest to maintain in your application. In this article I will explain:

How you can calculate these metrics and use them in your legacy project
How code coverage metrics can guide your technical debt's collection efforts

Churn

Churn by itself is a useful metric. It will tell you which files are the ones that are constantly changing. Change takes time, so files that are constantly changing are costing you time and money.

It will make you wonder:

Why are these files changing so much?
Are requirements constantly changing?
Are we truly understanding business rules correctly?
Is our application poorly designed?

If your legacy application is written in Ruby, you can use a gem called churn to find the files that are constantly changing.

Calculating Churn in Ruby

You can install churn like this:

gem install churn

Then call it like this:

churn -e rb --start_date "10 years ago" lib -i spec

This will show you a list of files sorted by their churn count (number of times that the file has changed in the SCM that you are using)

**********************************************************************
* Project Churn
**********************************************************************

Files
+-----------------------------------------------------------------------------------+---------------+
| file_path | times_changed |
+-----------------------------------------------------------------------------------+---------------+
| lib/metric_fu.rb | 50 |
| lib/metric_fu/version.rb | 44 |
| lib/metric_fu/configuration.rb | 36 |
| lib/metric_fu/reporting/templates/awesome/awesome_template.rb | 29 |
| lib/metric_fu/io.rb | 23 |
| lib/metric_fu/formatter/html.rb | 23 |
| lib/base/configuration.rb | 23 |
| lib/metric_fu/run.rb | 22 |
| lib/metric_fu/metrics/saikuro/saikuro.rb | 20 |
| lib/metric_fu/metric.rb | 20 |
+-----------------------------------------------------------------------------------+---------------+

By itself, this metric provides a limited insight:

These are the files that have changed the most since the beginning of the project

It doesn't provide insights into why these files are changing as much as they do. If you open these files in your editor, you might find that files with high churn are super complex or super simple.

Complexity

As you go about making your changes, you will introduce complexity. There is noway around it. The best way to have no complexity is to have no code.

It doesn't matter whether you are a rockstar object-oriented architect or a novice programmer. You will introduce complexity into a project.

"Complexity" should not be considered a negative term. "Unnecessary complexity" might be considered an anti-pattern. Good object-oriented programming calls for loose coupling and high cohesion.

You want modules that follow SOLID principles. When you notice your module is becoming extremely complex, you will want to refactor it into two or more modules. These modules will send and receive messages between each other, which will increase coupling (as loosely as possible) and hopefully improve cohesion per module.

As you go about submitting pull request to ship new features, change existing ones, or patch bugs, the maintainability of your project will vary according to the modules you introduce or change.

The bigger the project, the more complexity, the harder it will be to maintain it, because there will be more moving pieces and loosely coupled modules.

If you refactor a huge module, into two smaller modules, you will notice that cognitive complexity per module will (hopefully) get better. You won't have one huge, complex module anymore, you will have two or more modules that are easier to understand.

Modules that have one and only one responsibility will be easier to maintain and test. Future changes will be easier because you will be able to quickly understand what the module is doing.

In the next section I will explain how you can calculate complexity for all your Ruby modules.

Calculating Complexity in Ruby

There are many tools out there (e.g. three awesome libraries to assess code quality) that can help you find the most complex files in your project. One of the most popular tools in the Ruby world is flog.

Attractor uses flog to calculate complexity for each module. I really like the report that it generates. You can see all the modules in one page:

By itself this metric can take you in the wrong direction. Sometimes you will find extremely complex files which never change. So they are likely not your priority when it's time to refactor code.

Churn vs. Complexity

In his article, Michael Feathers, states the following:

These diagrams give us quite a bit of information. The upper right quadrant is particularly important. These files have a high degree of complexity, and they change quite frequently. [...] These classes are particularly ripe for arefactoring investment

I agree that this graph is great for finding refactoring candidates. The problem is that sometimes these candidates have no tests to verify their behavior. So sometimes we don't have time to:

Write tests
Refactor complex files

Sometimes we have to pick one or the other. So in my opinion it would be great to include another metric into our analysis: Code Coverage.

Code Coverage

When we start maintaining a module, we don't usually just look at its churn and complexity. We also look at the tests that describe its expected behavior.

Usually a module that has a decent test suite (basic code coverage) will be easier to understand than one without tests. We will have one more resource to learn about a module's behavior.

Code coverage metrics don't tell whether the test suite is good or bad. It tells you how many statements of your application are exercised by your test suite. You should not blindly trust code coverage percentages without doing a quick code review of the tests that are present.

Don't you just love it when a module has great tests that describe the expected behavior of its public API?

Churn vs. Complexity vs. Code Coverage

When I look at all the modules in the project, I want to know:

How many times has this file changed since the beginning of the project? (Churn)
How complex is this module? (Complexity)
What's the code coverage associated with this module? (Code Coverage)

This new dimension will tell us which files are the most changed, the most complicated, and the least covered with tests.

Refactoring modules that lack proper tests can quickly turn into a nightmare. You don't want your refactoring efforts to blow up in your face when changes hit production. And you probably don't want to test all your changes manually.

By adding a third dimension to the churn vs complexity graph, you will be able to gather new insights about your modules:

Should I refactor this module?
Should I increase test coverage before I refactor this module?
Do I have time to do both this week?

Calculating a Churn vs. Complexity vs. Code Coverage Graph in Ruby

Last year I published a library that can help you take all these metrics into account: Skunk. It relies on RubyCritic and SimpleCov to get all the metrics it needs.

You can install it like this:

gem install skunk

And run it like this:

skunk lib/

It won't generate a graph (yet) but it will generate a table with all the modules in the lib directory. It will sort files from worst to best (in terms of technical debt):

+-----------------------------------------------------+----------------------------+----------------------------+----------------------------+----------------------------+----------------------------+
| file | skunk_score | churn_times_cost | churn | cost | coverage |
+-----------------------------------------------------+----------------------------+----------------------------+----------------------------+----------------------------+----------------------------+
| lib/skunk/cli/commands/default.rb | 166.44 | 1.6643999999999999 | 3 | 0.5548 | 0 |
| lib/skunk/cli/application.rb | 139.2 | 1.392 | 3 | 0.46399999999999997 | 0 |
| lib/skunk/cli/command_factory.rb | 97.6 | 0.976 | 2 | 0.488 | 0 |
| test/test_helper.rb | 75.2 | 0.752 | 2 | 0.376 | 0 |
| lib/skunk/rubycritic/analysed_module.rb | 48.12 | 1.7184 | 2 | 0.8592 | 72.72727272727273 |
| test/lib/skunk/cli/commands/status_reporter_test.rb | 45.6 | 0.456 | 1 | 0.456 | 0 |
| lib/skunk/cli/commands/base.rb | 29.52 | 0.2952 | 3 | 0.0984 | 0 |
| lib/skunk/cli/commands/status_reporter.rb | 8.0 | 7.9956 | 3 | 2.6652 | 100.0 |
| test/lib/skunk/rubycritic/analysed_module_test.rb | 2.63 | 2.6312 | 2 | 1.3156 | 100.0 |
+-----------------------------------------------------+----------------------------+----------------------------+----------------------------+----------------------------+----------------------------+

The SkunkScore is a function of complexity and code coverage.

Let's say that your application has two extremely complex modules: user and product. Let's assume that both of them have changed a lot (churn count at 100) and are extremely complex (1,000 flog points).

In this example, if user has more tests than product, user's SkunkScore will be lower than product's. Next week you could start refactoring user or increasing code coverage for product

By looking at this table you will be able to quickly decide about what to do: You might want to write tests for a complex module, refactor a complex module that is well covered by integration tests, or both!

Code Coverage as a Prioritization Mechanism

In big projects you might notice hundreds of modules in the upper right quadrant of the Churn vs. Complexity graph. You might ask yourself: What do I do first? Do I write a test for module User? Do I refactor module Product?

I strongly believe that Code Coverage is a great signal for complementing churn and complexity metrics. I hope you can use it in your next legacy project to gradually pay off technical debt!

Our Guide for Unmaintained Open Source Projects

Ernesto Tagwerker (he/him) — Thu, 26 Mar 2020 11:16:00 +0000

There are some really great guides for starting a new open source projects, yet when it comes to dealing with a possibly abandoned, unmaintained project, there is no definitive guide for users, contributors, or maintainers.

I hope that this can be a useful guide for our community.

Problem

When do you declare that an open source project has been abandoned? How manydays have to go by until you start maintaining your own fork? What's thestandard for communicating with maintainers, contributors, and users? How doyou avoid n competing OSS forks of popular projects? How do you avoid duplicated work by people who want to maintain popular, but unmaintained OSS projects? What's the best way to find that one fork everybody is using?

Nights and Weekends

Too many applications out there are using libraries that depend on peoplehaving enough time during nights and weekends to work on their side projects.Most of the maintainers out there are not paid for their work. They might have_some help_ from their employers (e.g. Ombu Labs sponsors open source efforts), Tidelift, GitHub Sponsorships, Gitcoin, or OpenCollective, but most projects don't have financial support.

No wonder many OSS maintainers burn out after years maintaining their project.What happens when a maintainer burns out, starts using a different tech stack, or even worse, dies?

As users, we should be entirely grateful for their hard work, but we should also find a way to continue maintaining their OSS project, by starting a new fork or backing an existing one.

Proposal

Depending on your role in the project, here are some strategies that could help users and contributors, or maintainers:

For users and contributors: Continuing maintenance for an abandoned OSS project
For maintainers: Finding someone else to maintain your OSS project

Why

There is no clear way to continuing maintenance of an abandoned OSS project. There are definitely great guides that share a few important concepts, but not a lot of action steps for the case when an OSS project ends in maintenance limbo:

Advice for Open Source Users and Contributors

What can you do as someone who actively uses or contributes to the OSS project? For example: What can you do when you really really need to have support for Rails 6.0 for your application but this library is blocking you?

Best Judgement for Assessing Abandonment

First things first, use your best judgment to determine whether the project is truly abandoned or just feature complete.

There are many projects out there that are not being actively maintained because they do one little thing really well. Think about these projects as the outliers because they are feature complete and have no bugs.

Here are some things to consider when judging the state of an OSS project:

When was the last time there was a commit to the master branch?
When was the last time there was a commit to a non-master branch?
When was the last time there was a comment in an issue by a maintainer?
When was the last time there was a code review from one of the maintainers?
Is the test suite still passing if you submit one small change to the project?
When was the last version of the library published? (Example: for a Ruby gem,you can check in https://rubygems.org)
When was the last time an issue or pull request was closed by a maintainer?

Inquire

When it is not clear, it is okay to inquire about the status of the project. If someone hasn't done it already, you can submit an issue to the project asking about the state of things.

Before you submit your inquiry, you might want to submit a pull request with asmall improvement to the project. If the change is small enough and it improves things, then it should be easy to review or close. Make sure that the test suite passes in CI.

You can start your inquiry by saying that you appreciate their dedication and thank them for their hard work over the years, then you can ask if they're looking for help maintaining the project.

Do not demand an answer from the maintainers. Maintainers don't owe you anything because software is provided as is, so you will be lucky if you get a response to a nice request. So, please be nice.

You don't need to offer yourself as a potential maintainer for the project, maybe there are other people out there who are in a better position to continue maintaining the project.

If you don't get any response for a month, then you can move on to the next section.

Start Your Own Fork

You could start your own OSS project by creating a fork. Before you do that, make sure that you check the contribution network for the project. There might be someone else updating a fork that already has all the changes youneed.

Fun fact: Forking a project used to be considered a "bad word" but I think the rise of GitHub turned that around.

This is what Solidus did with Spree. A group of contributors got together and decided to start a new open source, e-commerce framework which started as a fork of Spree.

The governance model for Solidus is actually quite interesting and definitely a step in the right direction based on lessons learned from Spree and Solidus' history

Assuming you didn't skip any steps, you may want to post a comment in the issue you submitted initially. In there you can say that until the maintainers respond, you will be maintaining a fork of this project.

Advice for Open Source Maintainers

What can you do as someone who is maintaining an OSS project to make it easieron yourself? What can you do as someone who is done maintaining the projectand wants to move on? Here are some ideas based on my own experience and advice from other maintainers.

Look for Co-Maintainers

You don't have to do it alone. It's very likely that many of the users and contributors of your project would be happy to step up their commitment to co-maintain it. You can go so much further if you build a team of co-maintainers.

Nobody wants to spend their holidays answering comments, reviewing pull requests, or triaging bugs. The nice thing about finding co-maintainers is that they can step up when you can't.

A quick and simple way to do this is to post an issue to your repository. Another one is to post a notice at the beginning of the README. You don't have to accept all the people who are interested, but maybe some recent contributors will be interested.

Look for Someone to Continue Maintenance

If you can't actively maintain the project anymore, you can update your README sharing this with your users and contributors. You can say that your project needs a new maintainer and wait for people to reach out to you.

If nobody reaches out to you, then maybe they are using an alternative solution.You can update your README to mention that your project is no longer being maintained and that there are other alternatives to solve a similar problem.

Finally, if you are using GitHub, you can archive your repository. This will show a warning bar to anyone who gets to your project's page:

This repository has been archived by the owner. It is now read-only.

Vet Potential Maintainers or Co-Maintainers

Of course, as a responsible maintainer you don't want to let some random personon the Internet maintain your project. Trust needs to be earned. I think that's probably the most concerning part about giving up control.

Do you know this person? Do you know someone who knows them? Have they been contributing to your project for years? How can you really trust them?

Before you give a person permission to merge to master and publish new versions of your library, you can ask them to submit n contributions to your project. That way you can see if they know what they are doing and you trust them to steer the project in the right direction.

Send Me a Pull Request

Did I miss anything? Do you have any tips that could be useful to the OSS community?

This article is open source and you can find it here: https://github.com/fastruby/blog. So if you see something that could be improved or correct, please fork our repo and send a pull request. Thank you!

Bye bye, Medium! Hello, Jekyll 4.0! 👋🏼

Ernesto Tagwerker (he/him) — Tue, 03 Dec 2019 02:25:00 +0000

This year Medium decided to implement a paywall with all their content (meaning my content, and your content, and anyone’s content published in their platform)

When that happened I thought it was a good idea to leave their platform and use Jekyll instead. It took me almost 10 months to finally move away from Medium! This Thanksgiving break was a good time to spend a few hours migrating my content to Jekyll + GitHub Pages.

Here is how I did it:

I started with FastRuby.io’s blog’s source code (which is open source). Then I followed the instructions in this article: https://hackernoon.com/medium-2-md-convert-medium-posts-to-markdown-with-front-matter-c044e02c3cbb
I tweaked some of the content and file names. I had to massively edit file names so that Jekyll could figure out that it had to generate HTML pages for them.
I migrated from Jekyll 3.0 to 4.0. Fortunately the migration was quite easy! You can see all the changes over here: https://github.com/etagwerker/etagwerker.com/pull/1/files
I implemented Hyde and tweaked a few details to find a look & feel that I liked.
I configured Route 53 and GitHub to serve the domain without www. The nice thing about GitHub Pages is that it is simple, easy, and free for open source projects.

Here is the end result!

If you are thinking about migrating away from Medium, Jekyll is a great alternative to have full control of your platform and content! ❤️

PS: And if you want to take advantage of a great, developer-focused platform, you can cross-post all your articles to DEV! You just need to set up the right settings in your profile:

Introducing Skunk: Combine Code Quality and Coverage to Calculate a Stink Score

Ernesto Tagwerker (he/him) — Mon, 04 Nov 2019 10:00:00 +0000

Two weeks ago I had the opportunity to speak at Solidus Conf 2019.I presented Escaping the Tar Pitfor the first time and I got to talk about a few metrics that we can use toquickly assess code qualityin any Ruby project.

In this article I'd like to talk about Skunk: A Stink Score Calculator!I'll explain why we need it, how it works, and the roadmap for this new tool.

Every month we get contacted by leads (potential clients) who want to work withus on their Rails upgrade projects. Given that wehave some basic requirements for all of our new client projects, we want tocarefully analyze every project before we commit to working on it.

We analyze two very important aspects:

Code Coverage
Code Quality

For Code Coverage we like to use SimpleCov.For Code Quality we like to use RubyCritic.Both tools give us a few signals which tell us a story about the health of aRails application. We want to answer these questions:

Is it a dumpster fire?
Are we going to get ourselves stuck in the tar pit?
Is it a project that is easy to maintain?

Skunk is a Ruby gem that will combine codequality metrics from Reek;Flay;Flog; andSimpleCov to calculate a Stink Scorefor a file or set of files.

Skunk is a library built on top ofRubyCritic. It uses the cost valuefor each module:

module RubyCritic
  class AnalysedModule
    def cost
      @cost ||= smells.map(&:cost).inject(0.0, :+) +
                (complexity / COMPLEXITY_FACTOR)
    end
  end
end

The cost is a combination of smells and complexity:

Smells: They come from static code analysis performed by Flog; Flay; and Reek.
Complexity: It comes from Flog's total ABC metric

After determining that the cost, Skunk penalizes modules which lack code coverageby multiplying their cost by a factor directly related to the lack of coverage:

module RubyCritic
  # Monkey-patches RubyCritic::AnalysedModule to add a stink_score method
  class AnalysedModule
    PERFECT_COVERAGE = 100

    # Returns a numeric value that represents the stink_score of a module:
    #
    # If module is perfectly covered, stink score is the same as the
    # `churn_times_cost`
    #
    # If module has no coverage, stink score is a penalized value of
    # `churn_times_cost`
    #
    # For now the stink_score is calculated by multiplying `churn_times_cost`
    # times the lack of coverage.
    #
    # For example:
    #
    # When `churn_times_cost` is 100 and module is perfectly covered:
    # stink_score => 100
    #
    # When `churn_times_cost` is 100 and module is not covered at all:
    # stink_score => 100 * 100 = 10_000
    #
    # When `churn_times_cost` is 100 and module is covered at 75%:
    # stink_score => 100 * 25 (percentage uncovered) = 2_500
    #
    # @return [Float]
    def stink_score
      return churn_times_cost.round(2) if coverage == PERFECT_COVERAGE

      (churn_times_cost * (PERFECT_COVERAGE - coverage.to_i)).round(2)
    end
  end
end

After doing all these calculations, we get a Stink Score for the files we are evaluating:

$ skunk
running flay smells
.............
running flog smells
.............
running reek smells
.............
running complexity
.............
running attributes
.............
running churn
.............
running simple_cov
.............
New critique at file:////skunk/tmp/rubycritic/overview.html
+-----------------------------------------------------+----------------------------+----------------------------+----------------------------+----------------------------+----------------------------+
| file | stink_score | churn_times_cost | churn | cost | coverage |
+-----------------------------------------------------+----------------------------+----------------------------+----------------------------+----------------------------+----------------------------+
| lib/skunk/cli/commands/default.rb | 166.44 | 1.6643999999999999 | 3 | 0.5548 | 0 |
| lib/skunk/cli/application.rb | 139.2 | 1.392 | 3 | 0.46399999999999997 | 0 |
| lib/skunk/cli/command_factory.rb | 97.6 | 0.976 | 2 | 0.488 | 0 |
| test/test_helper.rb | 75.2 | 0.752 | 2 | 0.376 | 0 |
| lib/skunk/rubycritic/analysed_module.rb | 48.12 | 1.7184 | 2 | 0.8592 | 72.72727272727273 |
| test/lib/skunk/cli/commands/status_reporter_test.rb | 45.6 | 0.456 | 1 | 0.456 | 0 |
| lib/skunk/cli/commands/base.rb | 29.52 | 0.2952 | 3 | 0.0984 | 0 |
| lib/skunk/cli/commands/status_reporter.rb | 8.0 | 7.9956 | 3 | 2.6652 | 100.0 |
| test/lib/skunk/rubycritic/analysed_module_test.rb | 2.63 | 2.6312 | 2 | 1.3156 | 100.0 |
| lib/skunk.rb | 0.0 | 0.0 | 2 | 0.0 | 0 |
| lib/skunk/cli/options.rb | 0.0 | 0.0 | 2 | 0.0 | 0 |
| lib/skunk/version.rb | 0.0 | 0.0 | 2 | 0.0 | 0 |
| lib/skunk/cli/commands/help.rb | 0.0 | 0.0 | 2 | 0.0 | 0 |
+-----------------------------------------------------+----------------------------+----------------------------+----------------------------+----------------------------+----------------------------+

Stink Score Total: 612.31
Modules Analysed: 13
Stink Score Average: 0.47100769230769230769230769231e2
Worst Stink Score: 166.44 (lib/skunk/cli/commands/default.rb)

The most important signals here are:

Average Stink Score per module
Most complex files with little to no code coverage

We now know where we stand. We can clearly see the state of the application interms of code coverage and project complexity. We can now answer this question: "Which are the most complex files with the least coverage?"

We can use the Stink Score to guide us in our refactoring efforts:

How can I pay off technical debt and invest in the future of my application?
If I were to write tests to decrease the stink score, which files could Iwrite tests for?
If I were to refactor some of the most complex files, which files with goodcode coverage could I refactor?

Caveats

Skunk expects you to have a .resultset.json file in the coverage directorywithin the directory that you are evaluating. It uses the data within that fileto calculate the code coverage percentage for each module.

That means that you will have to run your test suite with SimpleCov enabled before you call skunk.

Total Stink Score is not a useful metric within a single project, as the totalwill continue to grow as you add more features to your application. It iscertainly a useful metric if you use it to compare two projects.

Known Issues

The calculation of the Stink Score is not 100% accurate. It is comparing amodule's code coverage and a module's complexity. It should be a method-basedcalculation: It should calculate the complexity of a method, the code coverageof the same method, then calculate the Stink Score per method.

Finally, the Stink Score of a module should be the sum of all the Stink Scoresin the module.

Roadmap

Assessing code quality for an application shouldn't stop at the applicationlevel. The Stink Score of our application is composed by two Stink Scores:

Stink Score of your application
Stink Score of your dependencies

Right now Skunk will only calculate Stink Score for your application code. Inthe future it should consider your dependencies as well, generating a StinkScore for each individual dependency.

The best way to assess progress in your project is to keep track of the StinkScore average over time. Is that number going up? Is it going down? How muchdoes your pull request change the Stink Score average? Right now Skunk doesnot support this, so you will have to do it manually.

Final Thoughts

I know that "stink" is a negative word to judge an application's technical debtand it might lead you down a negative path. By all means I don't want the StinkScore to be used in a witch hunt, to point fingers at code authors, or in anegative way in your team.

I seriously hope that you can use the Stink Score as the compass to move yourteam in the right direction. You should be able to use the Stink Score as acompass to gradually pay off technical debt:

Writing tests which increase code coverage will improve the Stink Score
Refactoring complex files will improve the Stink Score

Skunk will show you your location in the map of technical debt. It will alsoshow you a few paths to take to get to a better place. You will be able toprioritize the paths and pick one to pay off technical debt.

What do you think about this new metric for technical debt? Would you use itnext time you need to evaluate legacy code?

Please let me know in the comments below or come talk to me atRubyConf 2019 (I'll be speaking aboutthis topic at the conference)