How To Write Route53 Stubbed Responses For Rspec Tests

In this blog post, I will go over a recent exercise to fix some bugs, refactor, and write tests for some of our code related to Route53. Route53 is an AWS service that creates, updates, and provides Domain Name Service (DNS) for the internet. The reason that code unit tests are so important is because it helps reveal bugs, creates supportable and high quality code, and allows restructuring and refactoring with confidence. The downside to writing unit tests is that it can be time consuming, difficult at times, and bloating to the normal code base. It is not uncommon for unit tests’ "lines of code" (LOC) count to far exceed the LOC for the actual codebase. You would not be crazy to have nearly an order of magnitude difference in LOC for actual codebase versus LOC for unit test cases.

In this case, interacting with the AWS Route53 API was daunting to test and stubbing responses seemed incredibly difficult until I found some examples written by another one of our engineers that showed how the rspec and API SDKs could be made to work in a fairly straightforward and (dare I say) downright fun method for unit testing Ruby code.

The Code Under Examination

This straightforward code snippet was my first target for unit testing. It is very simple and only does one thing. It is ripe for refactoring for readability and reusability for other sections of the code. This should be the best way to begin the project and get familiar with the rspec templates I’d be using later. Before I start refactoring and fixing bugs, I wanted to write tests. Other than the fairly “inliney” and hard to follow syntax and “magical” code, can you spot any bugs?

def route53_hosted_zone_id(subdomain)
  route53.list_hosted_zones_by_name.map do |response|
    response.hosted_zones.detect{|zone| zone.name == "#{subdomain}." }&.id&.gsub(/.*\//, '')
  end.flatten.compact.first
end

Write Helpers Before the Refactor

I am already itching to remove the magical subdomain rewriting and gsub deleting into separate methods that can be reused and are easier to read:

def cannonicalise(hostname)
  hostname = domain_parts(hostname).join('.')

  "#{hostname}."
end

def parse_hosted_zone_id(hosted_zone_id)
  return nil if hosted_zone_id.blank?

  hosted_zone_id.gsub(%r{.*/+}, '')
end

Stub and Test the New Methods

First things first, we need to do a little bit of boilerplate to get the API calls mocked and stubbed, then add a few very simple tests to get started.

# frozen_string_literal: true

require 'rails_helper'

RSpec.describe Cloud::Aws::Route53 do
  let(:route53) { Aws::Route53::Client.new(stub_responses: true) }

  subject { FactoryBot.create(:v2_cloud_integration) }

  before do
    allow(subject).to receive(:route53).and_return(route53)
  end

  describe '#parse_hosted_zone_id' do
    context 'with a valid hostedzone identifier' do
      it 'returns just the zoneid' do
        expect(subject.parse_hosted_zone_id('/hostedzone/Z1234ABC')).to eq('Z1234ABC')
      end
    end
  end
  describe '#cannonicalise' do
    context 'without a dot' do
      it 'returns the zone with a dot' do
        expect(subject.cannonicalise('some.host')).to eq('some.host.')
      end
    end
    context 'with a dot' do
      it 'returns the zone with a dot' do
        expect(subject.cannonicalise('some.host.')).to eq('some.host.')
      end
    end
  end
end

Write A Fixture

Perfect, now we can test our new cannonicalise and parse_hosted_zone_id methods and we have a stubbed response coming from the Route53 API calls. Let’s write a simple new test to uncover some bugs by testing the api responses we get. The first step is to write some fixtures we can test with. Here we generate two faked stubbed responses for a very common domain.

context 'an AWS cloud integration' do
    before do
      route53.stub_responses(:list_hosted_zones_by_name, {
                               is_truncated: false,
                               max_items: 100,
                               hosted_zones: [
                                 {
                                   id: '/hostedzone/Z321EXAMPLE',
                                   name: 'example.com.',
                                   config: {
                                     comment: 'Some comment 1',
                                     private_zone: true
                                   },
                                   caller_reference: SecureRandom.hex
                                 },
                                 {
                                   id: '/hostedzone/Z123EXAMPLE',
                                   name: 'example.com.',
                                   config: {
                                     comment: 'Some comment 2',
                                     private_zone: false
                                   },
                                   caller_reference: SecureRandom.hex
                                 }
                               ]
                             })
    end
end

If you’re wondering how to make these fixtures, you can easily read the AWS Ruby SDK V3 documentation for sample inputs and outputs, or you can make API calls via the AWS CLI and inspect the responses, or you can even just put in some values and see what happens when you run rspec. For example, if I remove, say, the caller_reference parameter, I’ll get an error that helpfully identifies the problem.

You really can’t go wrong with the SDK validation and stubbed responses taken from the examples or from live requests you make with the CLI! This is already a tremendous benefit and we’re not even testing our own code yet.

Write a Test Case with the Stubbed Responses

Now we can write some unit test cases and loop through several responses that we expect to find the hosted zone. Voilá we’ve uncovered some bugs just by being a little creative with our inputs! Do you see why?

describe '#route53_hosted_zone_id' do
  %w[
    example.com
    example.com.
    www.example.com
    www.example.com.
    test.www.example.com
    test.www.example.com.
    deep.test.www.example.com
  ].each do |hostname|
    context 'for hosts that exist in the parent zone' do
      it "returns the hosted_zone_id for #{hostname}" do
        expect(route53).to receive(:list_hosted_zones_by_name).with(no_args).and_call_original
        hosted_zone_id = subject.route53_hosted_zone_id(hostname)
        expect(hosted_zone_id).to eq('Z123EXAMPLE')
      end
    end
  end
end

What these failed test cases are telling us is that the code worked under perfect conditions but in strange scenarios that may not be uncommon (for example, having an internal private zone and public zone with the same name, or selecting a two-level-deep name in a zone) could cause unpredictable behaviours.

The Solution is an Exercise for the Reader

Now we merely need to write or refactor the code from our original snippet to pass all of our new test cases. One of the issues that our test cases revealed was that two-level-deep names (say, test.www.example.com in the zone example.com) would be missed. We also needed a way to ensure that zones are not private, perhaps with an optional parameter to specify private zones. Here is an example that passes all the existing tests and welcome feedback on any other bugs or optimisations you find.

def route53_hosted_zone_ids_by_name(is_private_zone: false)
  # TODO: danger, does not handle duplicate zone names!!!
  hosted_zone_ids_by_name = {}
  route53.list_hosted_zones_by_name.each do |response|
    response.hosted_zones.each do |zone|
      if !!zone.config.private_zone == is_private_zone
        hosted_zone_ids_by_name[zone.name] = parse_hosted_zone_id(zone.id)
      end
    end
  end
  hosted_zone_ids_by_name
end

def route53_hosted_zone_id(hostname)
  # Recursively look for the zone id of the nearest parent (host, subdomain, or apex)
  hosted_zone_ids_by_name = route53_hosted_zone_ids_by_name

  loop do
    hostname = cannonicalise(hostname)
    break if hosted_zone_ids_by_name[hostname].present?

    # Strip off one level and try again
    hostname = domain_parts(hostname).drop(1).join('.')
    break if hostname.blank?
  end
  hosted_zone_ids_by_name[hostname]
end

Congratulations

All test cases now pass! Keep writing tests until you get nearly 100% coverage!