DEV Community

loading...
Cover image for Working with the AWS SDK for Ruby - Part II

Working with the AWS SDK for Ruby - Part II

pabloxio profile image Pablo Jaramillo ・5 min read

After clarifying in Part I the differences between Aws::EC2::Client, Aws::EC2::Resource and Resources (e.g. Aws::EC2::Vpc, Aws::EC2::Instance, etc), today we'll see how to add tests while we're using AWS SDK for Ruby. We'll follow the repo pabloxio/ruby_aws_sdk_rspec to understand how to stub responses for the AWS EC2 Client.

 Cloning repo

git clone git@github.com:pabloxio/ruby_aws_sdk_rspec
cd ruby_aws_sdk_rspec
bin/bundle install
Fetching gem metadata from https://rubygems.org/...
Using bundler 2.2.6
Fetching aws-partitions 1.418.0
Fetching aws-eventstream 1.1.0
Fetching jmespath 1.4.0
Fetching diff-lcs 1.4.4
Installing aws-eventstream 1.1.0
Installing jmespath 1.4.0
Installing aws-partitions 1.418.0
Installing diff-lcs 1.4.4
...
Installing aws-sdk-ec2 1.221.0
Bundle complete! 2 Gemfile dependencies, 13 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
Enter fullscreen mode Exit fullscreen mode

lib/list_ec2_instances.rb

ListEC2Instances is a small Ruby class that has four methods to retrieve EC2 instances information using the Aws::EC2::Client (#all_instances_using_client and #by_state_using_client) and Aws::EC2::Resource (#all_instances_using_resource and #by_state_using_resource) classes from AWS Ruby SDK. All methods return an Array and each element contains a Hash with the instance_id and the instance state name:

require 'aws-sdk-ec2'

class ListEC2Instances
  def initialize(params={})
    @client   = params[:client]   || Aws::EC2::Client.new
    @resource = params[:resource] || Aws::EC2::Resource.new(client: @client)
  end

  def all_instances_using_client
    @client.describe_instances.reservations&.first&.instances&.map do |i|
      {id: i.instance_id, state: i.state.name}
    end || []
  end

  def by_state_using_client(state="running")
    instances = @client.describe_instances.reservations&.first&.instances
    instances.select {|i| i.state.name == state}.map do |i|
      {id: i.instance_id, state: i.state.name}
    end
  end

  def all_instances_using_resource
    @resource.instances&.map do |i|
      {id: i.instance_id, state: i.state.name}
    end
  end

  def by_state_using_resource(state="running")
    instances = @resource.instances
    instances.select {|i| i.data.state.name == state}.map do |i|
      {id: i.data.instance_id, state: i.data.state.name}
    end
  end
end
Enter fullscreen mode Exit fullscreen mode

You'll need programmatic access to an AWS account for the next steps:

bin/bundle exec irb -r ./lib/list_ec2_instances.rb
Enter fullscreen mode Exit fullscreen mode
2.7.2 :001 > list = ListEC2Instances.new
 => #<ListEC2Instances:0x00007fc0a1ab2ac0 @client=#<Aws::EC2::Client>, @resource=#<Aws::EC2::Resource:0x00007fc0a18fb4e8 @client=#<Aws::EC2::Client>>>
2.7.2 :002 > list.all_instances_using_client
 => [{:id=>"i-0b007c42dbf9fc300", :state=>"stopped"}]
2.7.2 :003 > list.all_instances_using_resource
 => [{:id=>"i-0b007c42dbf9fc300", :state=>"stopped"}]
2.7.2 :004 > list.by_state_using_client
 => []
2.7.2 :005 > list.by_state_using_client("stopped")
 => [{:id=>"i-0b007c42dbf9fc300", :state=>"stopped"}]
2.7.2 :006 > list.by_state_using_resource("stopped")
 => [{:id=>"i-0b007c42dbf9fc300", :state=>"stopped"}]
2.7.2 :007 >
Enter fullscreen mode Exit fullscreen mode

 Running Tests

bin/rspec

ListEC2Instances
  #all_instances_using_client
    should return nil without instances
    should return all instances
  #by_state_using_client
    should return only running instances
    should return only stopped instances
    should return empty list without stopping instances
  #all_instances_using_resource
    should return empty without instances
    should return all instances
  #by_state_using_resource
    should return only running instances
    should return only stopped instances
    should return empty list without stopping instances

Finished in 0.0778 seconds (files took 0.99912 seconds to load)
10 examples, 0 failures
Enter fullscreen mode Exit fullscreen mode

It's important to note that there is no need for AWS programmatic access because of EC2 client stubbing

spec/list_ec2_instances_spec.rb

require 'list_ec2_instances'

RSpec.describe ListEC2Instances do
  let(:empty_client){ Aws::EC2::Client.new(stub_responses: true) }
  let(:full_client){
    client = Aws::EC2::Client.new(stub_responses: true)
    client.stub_responses(:describe_instances, {reservations: [
      instances: [
        {instance_id: "i-aaaaaaaaaaaaaaaaa", state: {name: "running"}},
        {instance_id: "i-bbbbbbbbbbbbbbbbb", state: {name: "running"}},
        {instance_id: "i-ccccccccccccccccc", state: {name: "stopped"}}
      ]
    ]})

    client
  }
...
Enter fullscreen mode Exit fullscreen mode

We're setting up here two EC2 Clients with stub_responses: true, in this way we can stub the EC2 client responses, which means the AWS SDK doesn't produce any network traffic and the client return stubbed (fake) data. If we don't provide specific client responses it will return empty data (empty Arrays, Hashes, etc) which is the case for empty_client.

For full_client we have stubbed a response for EC2 DescribeInstances API call and it will return 3 instances.

  context "#all_instances_using_client" do
    it "should return empty list without instances" do
      list = ListEC2Instances.new({client: empty_client})
      result = list.all_instances_using_client

      expect(result).to be_empty
    end

    it "should return all instances" do
      list = ListEC2Instances.new({client: full_client})
      result = list.all_instances_using_client

      expect(result.size).to eq(3)
    end
  end
Enter fullscreen mode Exit fullscreen mode

We send the stubbed client to the ListEC2Instances initialization object before invoking the *_using_client methods, simulating different scenarios with empty_client and full_client.

For *_using_resource methods, we still need to stub EC2 Client responses referencing the same empty_client and full_client objects:

  let(:empty_resource){ Aws::EC2::Resource.new(client: empty_client) }
  let(:full_resource){ Aws::EC2::Resource.new(client: full_client) }
Enter fullscreen mode Exit fullscreen mode
  context "#by_state_using_resource" do
    it "should return only running instances" do
      list = ListEC2Instances.new({resource: full_resource})
      result = list.by_state_using_resource

      expect(result.size).to eq(2)
    end

    it "should return only stopped instances" do
      list = ListEC2Instances.new({resource: full_resource})
      result = list.by_state_using_resource("stopped")

      expect(result.size).to eq(1)
    end

    it "should return empty list without stopping instances" do
      list = ListEC2Instances.new({resource: full_resource})
      result = list.by_state_using_resource("stopping")

      expect(result).to be_empty
    end
  end
Enter fullscreen mode Exit fullscreen mode

We can re-use the same EC2 client stub response because Aws::EC2::Resource#instances implementation it's just a wrapper around the Aws::EC2::Client#describe_instances method, so the underlying EC2 API call is the same. Snippet code from https://github.com/aws/aws-sdk-ruby:

gems/aws-sdk-ec2/lib/aws-sdk-ec2/resource.rb
Enter fullscreen mode Exit fullscreen mode
    def instances(options = {})
      batches = Enumerator.new do |y|
        resp = @client.describe_instances(options)
        resp.each_page do |page|
          batch = []
          page.data.reservations.each do |r|
            r.instances.each do |i|
              batch << Instance.new(
                id: i.instance_id,
                data: i,
                client: @client
              )
            end
          end
          y.yield(batch)
        end
      end
      Instance::Collection.new(batches)
    end
Enter fullscreen mode Exit fullscreen mode

It's important to note that you always need to stub AWS EC2 Client (Aws::EC2::Instance) API calls, it doesn't matter if you are using Aws::EC2::Resource or Resources (e.g. Aws::EC2::Vpc, Aws::EC2::Instance, etc) This is because Resource(s) offer higher levels of abstractions for AWS Services but they are built on top of the EC2 Client as a Ruby idiomatic wrapper to give us a more friendly developer experience, that's what an SDK is all about.

If we were using the aws-sdk-s3 gem instead of the aws-sdk-ec2 gem, the same concepts are applied:

s3_client.rb
Enter fullscreen mode Exit fullscreen mode
require 'aws-sdk-s3'

client = Aws::S3::Client.new(stub_responses: true)

client.stub_responses(:list_buckets, buckets:[{name: "bucket1"}, {name: "bucket2"}])

puts client.list_buckets.buckets
Enter fullscreen mode Exit fullscreen mode
$ ruby s3_client.rb
{:name=>"bucket1", :creation_date=>nil}
{:name=>"bucket2", :creation_date=>nil}
Enter fullscreen mode Exit fullscreen mode

The pattern is similar:

  • Create a stubbed client
  • Add specific client responses for later invoking them
  • Wait for empty responses for non-stubbed responses

And following Resource/Client relationship, We can re-use the same stubbed response for Aws::S3::Resource#buckets method:

s3_resource.rb
Enter fullscreen mode Exit fullscreen mode
require 'aws-sdk-s3'

resource = Aws::S3::Resource.new(stub_responses: true)

resource.client.stub_responses(:list_buckets, buckets:[{name: "bucket1"}, {name: "bucket2"}])

puts resource.buckets.map{|b| b.data.name}
Enter fullscreen mode Exit fullscreen mode
$ ruby s3_resource.rb
bucket1
bucket2
Enter fullscreen mode Exit fullscreen mode

Discussion (0)

pic
Editor guide