DEV Community: Rafael Montás

Automate deployments with GitHub Actions and Docker Swarm - CI/CD

Rafael Montás — Sun, 12 May 2024 04:00:00 +0000

This is the second part of a series of posts on deploying a Rails app to a Docker Swarm cluster on a remote VPS. In the first part, we set up the Swarm cluster on Hetzner Cloud and deployed a Rails app to it from our local machine. If you haven’t read it yet, you can find it here. In this post, we’ll automate the deployment process with GitHub Actions, so that every time you push your code to GitHub a CI/CD pipeline will build and deploy your app to the Swarm.

We are not going to cover linting and testing in this post, but if you want to add those steps to your pipeline, you can check out this PR merged into Rails to add a workflow that runs Brakeman, RuboCop, and the test suite on every push and pull request to the main branch of the repository.

Handle environment variables

Let’s review the stack file we used to deploy the app to the Swarm in the first part of this series.

version: '3.7'

services:
  web:
    image: <dockerhub_username>/swarm-demo:prod
    depends_on:
      - database
    ports:
      - 80:3000
    environment:
      - RAILS_MASTER_KEY=c51c56d33c031224c4678f44dd7eafa0
      - POSTGRES_USER=swarm_demo
      - POSTGRES_PASSWORD=production-secure-password
    deploy:
      placement:
        constraints:
          - node.labels.type == app

  database:
    image: postgres:15
    environment:
      - POSTGRES_USER=swarm_demo
      - POSTGRES_PASSWORD=production-secure-password
    volumes:
      - db_data:/var/lib/postgresql/data
    ports:
      - 5432:5432
    deploy:
      placement:
        constraints:
          - node.labels.type == database

volumes:
  db_data:

We have some environment variables in the stack file that we don’t want to expose in the repository but definitely need them to deploy the app. In order to handle this, we will use theBitwarden CLI to fetch the secrets from Bitwarden and inject them into the stack file when the CI/CD pipeline runs.

First, update the stack file so that it references an environment file instead of hardcoded values:

version: '3.7'

services:
  web:
    image: <dockerhub_username>/swarm-demo:prod
    depends_on:
      - database
    ports:
      - 80:3000
    env_file:
      - .env
    deploy:
      placement:
        constraints:
          - node.labels.type == app

  database:
    image: postgres:15
    env_file:
      - .env
    volumes:
      - db_data:/var/lib/postgresql/data
    ports:
      - 5432:5432
    deploy:
      placement:
        constraints:
          - node.labels.type == database

volumes:
  db_data:

Second, create a free Bitwarden accountif you don’t have one already and add RAILS_MASTER_KEY, POSTGRES_USER, and POSTGRES_PASSWORDas secure notes to the default collection:

Now, install the Bitwarden CLI by following the instructions here.

Once you have the Bitwarden CLI installed and authenticated, you can fetch the secured notes by running:

bw get notes <item-id>

Hydrate `.env` file with Bitwarden secrets

Wi will implement a rake task to fetch the secrets from Bitwarden and inject them into the .env file. This is a similar but simplified version of what Kamaldoes when you run kamal envify.

Create said rake task in lib/tasks/env_variables.rake:

namespace :env_variables do
  # "Creates .env by evaluating .env.erb"
  task :hydrate do
    File.write(".env", ERB.new(File.read(".env.erb")).result, perm: 0o600)
  end
end

Now, create a .env.erb file in the root of your project with the following content:

<% if (session_token=`bw unlock --passwordenv BW_PASSWORD --raw`.strip) != "" %>
RAILS_MASTER_KEY=<%= `bw get notes <item-id> --session #{session_token}` %>
POSTGRES_USER=<%= `bw get notes <item-id> --session #{session_token}` %>
POSTGRES_PASSWORD=<%= `bw get notes <item-id> --session #{session_token}` %>
<% else raise ArgumentError, "session_token token missing" end %>

With this setup, when you run rake env_variables:hydrate the .env file will be created as a result of ERB.new(File.read(".env.erb")).result executing the embedded Ruby code in the.env.erb file.

Now that we can dynamically inject the secrets into the .env file, let’s handle the CI/CD pipeline.

GitHub Actions

GitHub Actions is a continuous integration and continuous delivery (CI/CD) platform that allows you to automate your build, test, and deployment pipeline. GA uses the concept of a Workflow which is defined by a YAML file in the .github/workflows direcory.

We are going to configure a GitHub Actions workflow to be triggered when we push our changes to the repository, deploying the app to the VPS using Docker Swarm.

First, let’s create the .github/workflows directory and create a new file (workflow). Let’s call it deploy.yml and add the following code. We will break down the code right after.

Let’s break down the name, run-name, and on keys:

name: Deploy
run-name: Deploying to Swarm cluster
on:
  push:
    branches: [main]

The name key is the name of the workflow and the run-name key is the name for workflow runs, which will appear in the list of workflow runs in the GitHub Actions tab. The on key defines the events that trigger the workflow. In this case, the workflow will run when we push to the main branch.

The jobs key defines the jobs that will run in the workflow. In this case, we have only one job called deploy that runs on an ubuntu-latest runner.

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      ...

The steps key defines the steps that will run in the job. Each step is a task that can run commands or actions. The first step checks out the codebase, the second step sets up Ruby on the runner, the third step sets up Node.js so we can use the Bitwarden CLI, which is installed in the fourth step to hydrate the .env file. The fifth and sixth steps set up Docker Buildx and login to Docker Hub, respectively so that we can build and push the Docker image in the seventh step. Finally, the last step deploys the stack to the Swarm cluster.

In order to run the workflow, we need to add the secrets to the repository. Go to the repository settings, under Security , click Secrets and Variables and then click Actions. Add the following secrets:

Once you have set up the secrets, you can push your changes to the repository and the workflow should run automatically deploying the app to the Swarm cluster.

That’s it! You have successfully automated the deployment process of your Rails app to a Docker Swarm cluster on a remote VPS using GitHub Actions. Hopefullly, this post was helpful to you. If you have any questions or suggestions, feel free to reach out to me on Twitter.

Deploy a Rails app to a Docker Swarm cluster on Hetzner

Rafael Montás — Wed, 14 Feb 2024 04:00:00 +0000

In this post, I’ll show you how to set up a Docker Swarm cluster on Hetzner and deploy a Rails app to it. We will be setting up two hosts, one for the web server acting as manager and the other for the database acting as worker.

First, let’s create a new Rails app and prepare the database:

$ rails new swarm-demo -a propshaft -d postgresql -c tailwind
$ rails db:prepare

You should now be able to run the app locally with bin/dev and see the welcome page.

Just so we can test our database connection and data persistency in the swarm, let’s scaffold a simple Post resource and set the index action as the root route:

$ rails g scaffold Post title:string
$ rails db:migrate


# config/routes.rb
root "posts#index"

Provisioning the servers on Hetzner

We could do this through the Hetzner Cloud Console but we will use the hcloud CLI. First, you need to create an account on Hetzner, create a project, and generate an API token(with read and write permissions). Then, install the hcloud CLIwith brew install hcloud on macOS.

Once you have the token, you can authenticate with the CLI by creating a context:

$ hcloud context create swarm-hetzner

You will be prompted to enter the API token and the context will be created. Now, you can start provisioning the two servers.

In order for us to be able to SSH into the servers, we need to create an SSH key in our local machine and then associate it with Hetzner so we can use it when creating the servers:

For this example, I will create a new SSH key called id_rsa_personal without a passphrase.

$ ssh-keygen -f ~/.ssh/id_rsa_personal
$ hcloud ssh-key create --name personal-key --public-key-from-file ~/.ssh/id_rsa_personal.pub

You can list the SSH keys associated with Hetzner to make sure it was added successfully:

$ hcloud ssh-key list

ID NAME FINGERPRINT AGE
19362876 personal-key cf:0b:89:00:00:5f:7d:e5:93:c8:17:16:68:64:38:b3 55s

Now, we can create the two servers. We will use the cx11 type for both, which is the cheapest one. You can choose a different type if you want:

$ hcloud server create --name app-manager --type cx11 --image docker-ce --ssh-key 19362876
$ hcloud server create --name db-worker --type cx11 --image docker-ce --ssh-key 19362876

You should see the servers listed when you run hcloud server list:

$ hcloud server list

ID NAME STATUS IPV4 IPV6 PRIVATE NET DATACENTER AGE
43107995 app-manager running 37.27.34.249 2a01:4f9:c012:8df4::/64 - hel1-dc2 4m
43108038 db-worker running 95.217.128.149 2a01:4f9:c011:94ac::/64 - hel1-dc2 1m

Optionally, you can go on and add a new entry to the config file on your local device. With this, you will be able to simply use ssh <unique-name> instead of ssh <username>@<IP-address> to connect to the servers:

$ nano ~/.ssh/config

Host swarm-manager
  HostName 37.27.34.249
  User root
  IdentityFile ~/.ssh/id_rsa_personal
  PreferredAuthentications publickey

Host db-worker
  HostName 95.217.128.149
  User root
  IdentityFile ~/.ssh/id_rsa_personal
  PreferredAuthentications publickey

Now you can connect to the server with ssh swarm-manager and verify that Docker is installed:

$ ssh swarm-manager
$ docker --version

>> Docker version 25.0.1, build 29cf629

Adding firewalls to both servers

Now, let’s create our firewalls for the server and database. We will create the firewall for the web server first, add inbound rules, and then apply it to the server.

# Create the firewall
$ hcloud firewall create --name app-firewall
> Firewall 1245331 created

# Add inbound rules
$ hcloud firewall add-rule 1245331 --direction in --port 22 --protocol tcp --source-ips 0.0.0.0/0
$ hcloud firewall add-rule 1245331 --direction in --port 80 --protocol tcp --source-ips 0.0.0.0/0

# Docker related ports for nodes communication and overlay networks
$ hcloud firewall add-rule 1245331 --direction in --port 2377 --protocol tcp --source-ips 0.0.0.0/0
$ hcloud firewall add-rule 1245331 --direction in --port 7946 --protocol tcp --source-ips 0.0.0.0/0
$ hcloud firewall add-rule 1245331 --direction in --port 4789 --protocol udp --source-ips 0.0.0.0/0

For the database server, we will add the same rules but also allow traffic for the database port instead of the web server port:

# Create the firewall
$ hcloud firewall create --name db-firewall
> Firewall 1247136 created

# Add inbound rules
$ hcloud firewall add-rule 1247136 --direction in --port 22 --protocol tcp --source-ips 0.0.0.0/0
$ hcloud firewall add-rule 1247136 --direction in --port 5432 --protocol tcp --source-ips 0.0.0.0/0

# Docker related ports for nodes communication and overlay networks
$ hcloud firewall add-rule 1247136 --direction in --port 2377 --protocol tcp --source-ips 0.0.0.0/0
$ hcloud firewall add-rule 1247136 --direction in --port 7946 --protocol tcp --source-ips 0.0.0.0/0
$ hcloud firewall add-rule 1247136 --direction in --port 4789 --protocol udp --source-ips 0.0.0.0/0

Now we can apply the firewalls to the web server and database:

$ hcloud firewall apply-to-resource 1245331 --server 43107995 --type server
$ hcloud firewall apply-to-resource 1247136 --server 43108038 --type server

Setting up the Docker Swarm cluster

We will start by initializing Swarm mode on the manager node and then join the worker node to the cluster.

$ ssh swarm-manager
$ docker swarm init --advertise-addr 37.27.34.249

> Swarm initialized: current node (xyivupj1vs8j972fb789t3n08) is now a manager.
> To add a worker to this swarm, run the following command:
  docker swarm join --token SWMTKN-1-2z9r0ypycjcxesjr7vlyf4nd248xp7h83dy7127miwoyydszct-a04m5zkf3f6hd3ueeoj513tby 37.27.34.249:2377

Before joining the worker node to the cluster, let’s assign a label to the manager node so we can control where the app container will be deployed. First, list the nodes to get the manager node ID and then assign the label to the manager node:

$ docker node ls
$ docker node update --label-add type=app xyivupj1vs8j972fb789t3n08

Now, we can join the worker node to the cluster:

$ ssh db-worker

$ docker swarm join --token SWMTKN-1-2z9r0ypycjcxesjr7vlyf4nd248xp7h83dy7127miwoyydszct-a04m5zkf3f6hd3ueeoj513tby 37.27.34.249:2377

> This node joined a swarm as a worker.

Assign a label to the worker node as well by going back to the manager node, list the nodes so you can get the worker node ID, and then assign the label to the worker node:

$ ssh swarm-manager
$ docker node ls
$ docker node update --label-add type=database wqb9dnx6otanxoc529qfwblvq

Deploying the Rails app to the cluster

Since Rails 7.1, the rails new command generates a Dockerfile tuned for production use with proper caching layers and multi-stage building. We will use this file to build the image that will be used by the web service in the stack.

We will use a docker-stack.yml file to define the services and deploy the app to the cluster. Create the file in the root of the Rails app and add the following content:

version: '3.7'

services:
  web:
    image: <dockerhub_username>/swarm-demo:prod
    depends_on:
      - database
    ports:
      - 80:3000
    environment:
      - RAILS_ENV=production
      - RAILS_LOG_TO_STDOUT="true"
      - SECRET_KEY_BASE=b88d584663a98347075b76bf088a783f7c679e80f793519c2548674eaf0964d70db93b8167e44b1ecc767f02900a3aefb797bfe74c36943711153a408ef9554b
      - RAILS_SERVE_STATIC_FILES="true"
      - DATABASE_HOST=database
      - POSTGRES_USER=swarm_demo
      - POSTGRES_PASSWORD=production-secure-password
    deploy:
      placement:
        constraints:
          - node.labels.type == app

  database:
    image: postgres:15
    environment:
      - RAILS_ENV=production
      - RAILS_LOG_TO_STDOUT="true"
      - SECRET_KEY_BASE=b88d584663a98347075b76bf088a783f7c679e80f793519c2548674eaf0964d70db93b8167e44b1ecc767f02900a3aefb797bfe74c36943711153a408ef9554b
      - RAILS_SERVE_STATIC_FILES="true"
      - DATABASE_HOST=database
      - POSTGRES_USER=swarm_demo
      - POSTGRES_PASSWORD=production-secure-password
    volumes:
      - db_data:/var/lib/postgresql/data
    ports:
      - 5432:5432
    deploy:
      placement:
        constraints:
          - node.labels.type == database

volumes:
  db_data:

Make sure to replace the SECRET_KEY_BASE with the output of rails secret. Also, this is for demonstration purposes only, make sure not to expose sensitive information in your docker-stack.yml file.

In config/database.yml we need to set the host to database as it is the name of the service in the stack. Also, we need to set the username and password to the ones we defined in the docker-stack.yml file:

production:
  <<: *default
  database: swarm_demo_production
  host: database
  username: swarm_demo
  password: <%= ENV["POSTGRES_PASSWORD"] %>

Starting from Rails 7.1 ssl is enforced by default, so we need to set theconfig.force_ssl = false in the config/environments/production.rb file.

Build x86 images on m1 mac:

$ docker buildx build --platform=linux/amd64 -f Dockerfile -t <dockerhub_username>/swarm-demo:prod .

Upload image to registry:

$ docker push <dockerhub_username>/swarm-demo:prod

Now, we are ready to deploy the app to the cluster by creating a Docker Context for the manager node, switching to it, and then running the docker stack deploy command:

$ docker context create swarm-demo --docker host=ssh://37.27.34.249
$ docker context use swarm-demo
$ docker stack deploy -c docker-stack.yml swarm-demo

> Creating network swarm-demo_default
> Creating service swarm-demo_database
> Creating service swarm-demo_web

Now, you should be able to access the Rails app by visiting the IP address of the manager node in your browser:

To make shure the database is working, create a new post and verify that it persists after modifying the page’s header, rebuilding the image and redeploying the stack:

$ docker buildx build --platform=linux/amd64 -f Dockerfile -t <dockerhub_username>/swarm-demo:prod .
$ docker push <dockerhub_username>/swarm-demo:prod
$ docker stack deploy -c docker-stack.yml swarm-demo

I hope this post was helpful and that you were able to deploy your Rails app to a Docker Swarm cluster on Hetzner. If you have any questions or suggestions, feel free to reach out to me on Twitter.

Ruby’s method-lookup path and Object Individuation

Rafael Montás — Tue, 13 Jun 2023 04:00:00 +0000

Objects, in Ruby, don’t “have” methods but, rather, find them by searching its class and that class’s superclass, and onward, up to the Object or even BasicObject class, or in a module that has been mixed into any of those classes.

Most of the time, you’ll use the dot operator “.” (There’s also send, __send__, and public_send alternatives) to “prompt” the object to go find a method (send messages to objects). In practice, the message being sent is the name of a method.

Demonstration of basic module inclusion and inheritance

Let’s write some basic classes, modules, and methods so we can easily visualize the logic and mechanics of method lookup and hopefully better understand how objects find methods.

module M
  def hello
    puts "Hello from module M"
  end
end

class C
  include M
end
class D < C; end

object = D.new
object.hello

# Hello from module M
# => nil

The search ends when the method being searched for is found, or with an error condition if it isn’t found. This error condition is triggered by the method_missing method.

The Kernel module provides an instance method called method_missing. This method is executed whenever an object receives a message that doesn’t match a method anywhere in the object’s method-lookup path.

How far does the method search go?

However many classes and modules it may cross along the way, the search for a method can always go as far up as BasicObject, which has a few instance methods. But to understand the common behavior of all Ruby objects, you have to look at Object or more precisely, you have to look at Kernel where most of Ruby’s fundamental methods are defined. And because Object mixes in Kernel, all instances of Object and all descendants of it have access to the instance methods in Kernel.

The illustration below show the method search path for the code example above all the way up the chain.

Overriding methods

Since the method search process passes through multiple classes and modules, an object can have multiple methods with the same name in its method-lookup path. Still, if the object’s method-lookup path includes two or more same-named methods, the first one encountered is executed.

module Interest
  def calculate_interest
    puts "We are in module Interest"
  end
end

class Account
  include Interest

  def calculate_interest
    puts "We are in class Account"
  end
end

account = Account.new
account.calculate_interest

# We are in class Account
# => nil

Also, a class could mix in two or more modules and more than one implements the same method being searched for. In that case, the most-recently mixed-in module is searched first.

How `prepend` and `extend` work

Even though include is the most common way of mixing in modules into a class, Ruby provides two other ways to achieve that but with some differences.

prepend

If you prepend a module to a class, the object looks in that module first, before it looks in the class. it basically inserts the module at the beginning of the ancestors chain.

You can see the difference between include and prepend reflected when calling ancestors on the Person class, which lists all the classes and modules where an instance of the class will search for methods.

module LookHereFirst; end
module LookHereSecond; end

class Person
  prepend LookHereFirst
  include LookHereSecond
end

Person.ancestors

# => [LookHereFirst, Person, LookHereSecond, Object, PP::ObjectMixin, Kernel, BasicObject]

You could use prepend when you want methods in a module to take precedence over the versions defined in a given class.

extend

On the other hand, extend is another way of mixing a module into a class. The difference is that the module’s methods will be available as class methods instead of instance methods.

Had we used extend rather than prepend in our example above, the LookHereFirst module would not have been inserted into Person’s ancestors chain. Instead, Ruby inserts the module in the ancestors chain of Person’s singleton class.

Where do singleton methods fit in the method-lookup path?

An object’s singleton methods live in the object singleton class and so an object can call instance methods from its class and from its singleton class. It has both.

To solve a message into a method, an object looks in all the instance methods defined in these two classes, along with methods available through ancestral classes or through any modules that have been mixed in or prepended to any of these classes.

Let’s prepend and include two modules in an object’s singleton class and then update our diagram to see singleton classes taken into account in the method-lookup path.

The class << object notation is a common way of opening an object’s singleton class.

module M
  def hello
    puts "Hello from module M"
  end
end

module X
  def hello
    puts "Hello from module X in object's singleton class"
  end
end

module N; end
module Y; end

class C; end

class D < C
  prepend M
  include N
end

object = D.new

class << object
  prepend X
  include Y
end

object.hello

# Hello from module X in object's singleton class
# => nil

In its search for the method hello, object looks first for any module prepended to its singleton class; then it looks in the singleton class itself. It then looks in any modules that the singleton class has included. Finally, the search proceeds up to the object’s original class, and so forth.

Class methods are singleton methods

Class methods are singleton methods defined on objects of class Class. Normally, when you define a singleton method on an object, no other object can serve as the receiver in a call to that method. But, methods defined as singleton methods of a class object can also be called on subclasses of that class.

The diagram above shows the relationship among classes and their singleton classes.

In summary, now we better understand how an object looks for a method when resolving a message.

I really hope you find this article useful and thank you for taking the time.

Getting to know Ruby’s Enumerable Module

Rafael Montás — Sun, 16 Apr 2023 20:30:43 +0000

Enumerable is a module in the Ruby standard library. Classes that include it have to define an instance method called each, which yields the elements of the collection in succession. Once the iterator is defined and Enumerable is mixed in, the class now supports all sorts of collection-related behavior. Also, keep in mind, that although Enumerable adds common functionality to each of the collection classes that includes it, each of them actually overrides some of the methods.

class MyArray
  include Enumerable

  def each
    # yield elements of the collection
  end
end

You can query which methods Enumerable provides by sending the messageinstance_methods to it and passing the false argument so you get only methods defined in the module itself.

irb(main):001:0> Enumerable.instance_methods(false)

In order to understand Enumerable at a deeper level, let’s write a few of its methods while trying to cover as many operations as posible.

Methods for Querying

Some Enumerable methods return information about the collection other than the elements themselves. Some return either true or false if one or more element matches certain criteria.

#include?

Returns true if the item yielded to the block is equal the argument passed, falseotherwise.

def include?(value)
  self.each { |item| return true if item == value }
  return false
end

# (1..5).include?(2)
# => true

This works just fine for Arrays but if we were querying a Hash, we would need to adjust for the fact that when you iterate through a Hash with each it yields back one key/value pair at a time (two-element arrays). The Hash#include? method checks for key inclusion

#any?

Returns true if any element meets a specified criteria, false otherwise.

def any?
  return self.length >= 1 unless block_given?

  self.each do |element|
    return true if yield(element)
  end

  return false
end

# => (1..5).any? { |n| n > 3 }
# => true

#count

Returns the count of elements based on a criteria passed as argument or a block, if given.

def count(value=nil)
  counter = 0
  self.each do |element|
    if value
      counter += 1 if element == value
    elsif block_given?
      counter += 1 if yield(element)
    else
      counter += 1
    end
  end

  warn("warning: given block not used") if value && block_given?
  return counter
end

# (1..5).count { |value| value > 3 }
# => 2

#tally

Counts the ocurrences of each element. Returning a Hash with the collection’s elements as keys and corresponding counts as values.

def tally
  tally = Hash.new(0)
  self.each { |element| tally[element] += 1 }

  return tally
end

# [1, 2, 2, 2, 3, 3, 4].tally
# => {1=>1, 2=>3, 3=>2, 4=>1}

Methods for Searching and Filtering

It’s common to want to filter a collection of objects based on a selection criteria. We’ll look at several facilities for filtering and searching collections. All of them expect a code block, where you difine your selection criteria (your tests for inclusion or exclusion).

#find aliased as #detect

Returns the first element in the collection for which the code block returns true. If no code block is provided, it returns an instance of Enumerator.

def find
  return self.enum_for( __method__ ) unless block_given?
  self.each { |element| return element if yield(element) }

  return nil
end

alias :detect :find

# (1..5).find {|value| value > 2}
# => 3

#find_all aliased as #select

Returns an Array containing all the elements of the original collection that matched the criteria in the code block. If no matching elements are found, it returns an empty collection. Also, if no code block is provided, it returns an instance of Enumerator.

def find_all
  return self.enum_for( __method__ ) unless block_given?

  matches = []
  self.each do |element|
    matches << element if yield(element)
  end

  return matches
end

alias :select :find_all

# (1..9).find_all {|value| value % 3 == 0 }
# => [3, 6, 9]

#find_index

Returns the index of the first element that meets the specified criteria, or nil if no element matches the criteria. It returns an instance of Enumerator if neither a code block nor an argument are provided.

For this method to work, make sure your #each implementation returns anEnumerator when no code block is provided. This will allow us to chain methods, as in .each.with_index.

def find_index(value=nil)
  return self.enum_for( __method__ ) unless value || block_given?

  self.each.with_index do |element, index|
    if value
      return index if element == value
    else
      return index if yield(element)
    end
  end

  return nil
end

# (1..5).find_index(2)
# => 1

#group_by

When given a code block, it returns a Hash. For each unique value returned by the block, the results hash gets a key; the value for that key is an Array of all the elements of the collection for which the block returned that value. If no code block is given, it returns an instance of Enumerator.

def group_by
  return self.enum_for( __method__ ) unless block_given?

  groups = Hash.new([])
  self.each do |element|
    groups[yield(element)] << element
  end

  return groups
end

# colors = [{group: 'primary'}, {group: 'secondary'}, {group: 'primary'}]
# colors.group_by { |value| value[:group] }
# => {"primary"=>[{:group=>"primary"}, {:group=>"primary"}], "secondary"=>[{:group=>"secondary"}]}

#map aliased as #collect

Returns an Array when given a block. Returns and Enumerator otherwise. The returned array is always the same size as the original collection. Its elements are the result of calling the code block on each element in the original object.

def map
  return self.enum_for( __method__ ) unless block_given?

  collection = []
  self.each { |element| collection << yield(element) }

  return collection
end

alias :collect :map

# (1..5).map { |value| value * value }
# => [1, 4, 9, 16, 25]

#inject aliased as #reduce

Works by initializing an accumulator object, performs a calculation on each iteration and reset the accumulator to the result of that calculation. Returns the return value from the last block call.

def inject(operand=0)
  accumulator = operand
  self.each do |element|
    accumulator = yield(accumulator, element)
  end

  return accumulator
end

alias :reduce :inject

# (1..5).inject { |sum, value| sum + value }
# => 15

We’ve gotten to know Enumerable in a more detailed way and it will better positioned us to handle everyday programming as we’ll use the enumeration-related facilities of the language virtually every time we write a Ruby program.

Implementing Data Structures in Ruby: Arrays

Rafael Montás — Sat, 11 Mar 2023 19:43:57 +0000

Just recently, I transitioned careers to become a Software Developer using mainly Ruby and Ruby on Rails and wanted to strengthen my understanding of Data Structures & Algorithms, which I am sure would allow me to become a better programmer. So with this in mind, I decided to read The Well-Grounded Rubyist, Third Edition to better understand the basics of the Ruby language and A Common-Sense Guide to Data Structures and Algorithms, Second Edition to start implementing algorithms with Space and Time Complexity in mind.

While working through these books and a few blog posts from amazing people, I learned how to implement data structures from scratch in Ruby and in this post, we will do just that by trying to replicate the inner workings and methods of the array

What is an Array?

When a program declares an array, it allocates a contiguous set of empty cells for use in the program (Ruby implements dynamic arrays). This translates to an ordered collection of objects.

Ruby provides us with the class Array which we can instantiate, among other ways, by explicitly calling Array.new or by using the literal constructor [].

Building an Array from scratch

Let's begin by creating and initializing our class MyArray using a Hash to keep track of the elements within the collection. Also, Arrays keep track of the number of objects the collection contains at all times. We will implement an attr_reader to be able to query instances of MyArray about this information using MyArray#length.

class MyArray
  attr_reader :length

  def initialize(enum = nil)
    @length = 0
    @elements = Hash.new(nil)

    return if enum.nil?

    enum.each do |val|
      @elements[@length] = val
      @length += 1
    end
  end
end

We initialize our class with two instance variables, @length, set to 0 by default and @elements which is set to an empty Hash. Also, with this implementation, we can create a new MyArray containing the elements of a given enumerable object, increasing the @length on each iteration.

Before we move on, let's replicate even more the behavior and terminal output by creating a method that will allow us to instantiate our class like the way we instantiate arrays with literal constructors and specially similar to the class Set.

def self.[](*ary)
  new(ary)
end

def inspect
  sprintf('#<%s: [%s]>', self.class, @elements.values.inspect[1..-2])
end

alias to_s inspect

self.[](*ary) defines a class method and thanks to Ruby's syntactic sugar, we can now create instances of our class by calling MyArray['foo', 'bar']. The implementation of inspect was inspired by the way the Set class implements it. Resulting in a terminal output as shown below.

irb(main):001:0> MyArray['foo', 'bar']
=> #<MyArray: ["foo", "bar"]>

Now, let's start creating the methods we are going to need in order to cover the basic operations of reading, searching, inserting and deleting.

#[] (aliased as #slice)

When you have objects in an array you can retrieve those objects by using the #[] or #slice(index) methods. Retrieving single elements like this takes constant time in terms of Time Complexity analysis: O(1).

# Returns the element at the given index.
def [](index)
  @elements[index]
end

alias slice []

#[]=(index, element)

Let's now implement its setter equivalent #[]=(index, element) which allows us to set an element at a given index. In the case the given index is out of range for MyArray we set those slots between the given index and the last element to nil.

def []=(index, element)
  while index > @length
    @elements[@length] = nil
    @length += 1
  end
  @elements[index] = element
  @length += 1 if index == @length
end

#push(element) (aliased as #<<)

To add an element to the end of the collection, you can use #push(element) or #<<. Returning the collection itself. Here, knowing arrays have zero based indexing, we can implement it by setting the given element at the @length index, which is one slot to the right of the last element in the collection.

# Constant Time to add element to the end of collection
# Time Complexity: O(1)
def push(element)
  @elements[@length] = element
  @length += 1
  return self
end

alias << push

#pop

Here we remove the last element of the collection, reduce the @length instance variable by 1, and return the deleted element back which is accomplished by saving said element in a variable before removing it.

# Constant time to remove last element. Time Complexity: O(1)
def pop
  latest_element = @elements[@length -1]
  @elements.delete(@length -1)
  @length -= 1

  return latest_element
end

#unshift(value) and #shift

As you have seen, appending and removing elements from the end of the collection is fairly straight forward. Now, to accomplish the same operations at the beginning of the collection, we have to shift the remaining elements either to the right (if we are appending #unshift(value)) or to the left (if we are removing #shift). So, we will be creating a couple of private methods that will help us do just that.

#unshift(value)

class MyArray
  ...

  # Linear time to add at the beginning of collection.
  # Moves other values. Time Complexity: O(n)
  def unshift(value)
    self.unshift_elements(0, value)
    return self
  end

  private

  def unshift_elements(index, value)
    counter = @length - 1
    while counter >= index
      @elements[counter + 1] = @elements[counter]
      counter -= 1
    end
    @elements[index] = value
    @length += 1
  end
end

Here, when we call #unshift(value), we start shifting by 1 index to the right all elements of the collection, starting from the last one so we don't lose any value in the process. Once all elements are shifted, we set the element at index 0 to the value being inserted and increment the collection's length by 1.

#shift

Similarly, to remove the first element from the collection, we need to shift all elements but this time to the left. Basically, we start shifting left from index 1 until we have shifted the last element which will be duplicated, so we remove it and decrease our collection's length by 1.

class MyArray
  ...

  # Linear time to remove at the beginning of the collection.
  # Shifts other values. Time Complexity: O(n)
  def shift
    first_element = @elements[0]
    self.shift_elements(0)
    return first_element
  end

  private

  ...

  def shift_elements(index)
    counter = index
    while counter < (@length - 1)
      @elements[counter] = @elements[counter + 1]
      counter += 1
    end
    @elements.delete(@length - 1)
    @length -= 1
  end
end

Iterating over `MyArray`

In order to iterate over the collection, we need to implement the #each method. The idea behind #each is simple: by calling it on #MyArray collection it will yield back each item in it to the code block provided, one at a time. The return value of #each, when it’s given a block, is its receiver, the original collection. When it isn’t given a block, it returns an enumerator, which is what allows us to chain other methods like Enumerator#with_index.

# Calls the given block once for each element in the collection, passing
# the element as a parameter. Return an enumerator if no block is given.
def each
  return self.enum_for(__method__) unless block_given?

  counter = 0
  while counter < @length
    yield @elements[counter]
    counter += 1
  end

  return self
end

Having done that, we can implement the method #delete, which makes use of #each by first finding the index of the given element, removing it from the collection, and then shifting all remaining elements to the left.

#delete(element)

# Linear time to remove at an arbitrary location. Shifts other values.
# Time Complexity: O(n)
def delete(element)
  self.each.with_index do |value, index|
    if value == element
      self.shift_elements(index)
      return value
    end
  end

  return nil
end

The journey of learning the basics of Ruby and the inner working of collection classes like the Array has been an amazing experience and I'm looking forward to share all my learnings in subsequent posts like this one. I'll let you with the code for our entire MyArray class below and a link to the Github repo:

class MyArray
  attr_reader :length

  def initialize(enum = nil)
    @length = 0
    @elements = Hash.new(nil)

    return if enum.nil?

    enum.each do |val|
      @elements[@length] = val
      @length += 1
    end
  end

  def self.[](*ary)
    new(ary)
  end

  def inspect
    sprintf('#<%s: [%s]>', self.class, @elements.values.inspect[1..-2])
  end

  alias to_s inspect

  # Returns the element at the given index.
  def [](index)
    @elements[index]
  end

  alias slice []

  def []=(index, element)
    while index > @length
      @elements[@length] = nil
      @length += 1
    end
    @elements[index] = element
    @length += 1 if index == @length
  end

  # Constant Time to add element to the end of collection
  # Time Complexity: O(1)
  def push(element)
    @elements[@length] = element
    @length += 1
    return self
  end

  alias << push

  # Constant time to remove last element. Time Complexity: O(1)
  def pop
    latest_element = @elements[@length -1]
    @elements.delete(@length -1)
    @length -= 1

    return latest_element
  end

  # Linear time to add at the beginning of collection.
  # Moves other values. Time Complexity: O(n)
  def unshift(value)
    self.unshift_elements(0, value)
    return self
  end

  # Linear time to remove at the beginning of the collection.
  # Shifts other values. Time Complexity: O(n)
  def shift
    first_element = @elements[0]
    self.shift_elements(0)
    return first_element
  end

  # Calls the given block once for each element in the collection, passing
  # the element as a parameter. Return an enumerator if no block is given.
  def each
    return self.enum_for(__method__) unless block_given?

    counter = 0
    while counter < @length
      yield @elements[counter]
      counter += 1
    end

    return self
  end

  # Linear time to remove at an arbitrary location. Shifts other values.
  # Time Complexity: O(n)
  def delete(element)
    self.each.with_index do |value, index|
      if value == element
        self.shift_elements(index)
        return value
      end
    end

    return nil
  end

  private

  def unshift_elements(index, value)
    counter = @length - 1
    while counter >= index
      @elements[counter + 1] = @elements[counter]
      counter -= 1
    end
    @elements[index] = value
    @length += 1
  end

  def shift_elements(index)
    counter = index
    while counter < (@length - 1)
      @elements[counter] = @elements[counter + 1]
      counter += 1
    end
    @elements.delete(@length - 1)
    @length -= 1
  end
end

Sources of Inspiration

Data Structures From Scratch: Array
Exploring Data Structures From a Ruby Background, Pt. 1: Arrays
Ruby's Set source code