DEV Community: Cody Norman

Advanced Route Constraints with Rails

Cody Norman — Fri, 08 Jul 2022 14:51:02 +0000

Recently, I was working on a feature sending sms messages with appointment links. The links ended up taking up most of the message body so I wanted to look into some options to shorten the link to conserve space in the body of the
message.

Instead of using something like Bitly, I wanted to see if there were any simple options for something that would remain within the app. I did some googling for link shortener gems and tried a couple of the first ones
that popped up.

I was able to take my links and create a shortened version that gets sent out in the sms confirmation pretty easily. Everything seemed to be going fine until I noticed some test failures. Some of my static pages were returning 404. After a bit of digging, I noticed that all my static pages, like /about and /contact were being intercepted by the shortener.

When it didn't find a shortened link with an id of about it was returning a 404 and breaking my static pages.

The easiest solution would have been to just mount the shorteners rails engine under a prefix, something like /srt.

This is what I ended up with but wanted to attempt to solve the issue using route constraints.

Route constraints are something I've heard of, but have never had cause to use. I thought this would be a good chance to take a stab at routing constraints and get some more experience with them.

There are a few different options for route constraints. Some of the advanced options are using a lambda in the route definition and one that's a little more involved that uses a class for the route constraint.

My first approach was using a lambda within the route definition but eventually moved to using a class.

Rails Guides on Advanced Route Constraints

lambda route constraint example:

  get '*path', to: 'restricted_list#index',
    constraints: lambda { |request| RestrictedList.retrieve_ips.include?(request.remote_ip) }

class route constrain example:

class ShortenerRouteConstraint
  def matches?(request)
    # if request.path is not in static routes it should be a short link
    static_page_routes.exclude?(request.path)
  end

  private

  def static_page_routes
    # dynamically pull all static pages + admin, 404, and errors
    static_page_paths = Dir.new("app/views/static").children.map do |f|
      "/#{File.basename(f, ".html.erb")}"
    end
    static_page_paths << "/admin"
    static_page_paths << "/404"
    static_page_paths << "/500"
  end
end

My first pass had me adding the routes of my static pages to an array, and checking the request.path to see if it was included in the request.

That worked just fine, but after I added a new static page, I got another test fail for 404 errors (this happens when the shortener can't find an object).

I wanted to see if there was a way to make that check dynamic and keep from updating manually, usually after seeing a spec fail and having to refresh my memory on what's happening.

I created a route constrain class matching the one above. After having the route constraint class, the next step was to update the routes to use that constraint class.

# config/routes.rb
require 'app/services/ShortenerRouteConstraint.rb'

get "/:id" => "shortener/shortened_urls#show", constraints: ShortenerRouteConstraint.new

Note: I really didn't know the best place to put that class, so just defaulted to putting it in my app/services directory and requiring that file at the top of my config/routes.rb

Checking the routes against a dynamic list of view files is absolutely overkill,
but wanted to see if I could find a way to set and forget.

After a few minutes skimming through the Ruby docs and found what I needed.

  # dynamically pull all static pages
  static_page_paths = Dir.new("app/views/static").children.map do |f|
    "/#{File.basename(f, ".html.erb")}"
  end

This creates a new Dir object for the app/views/static directory. We access its children, and use map to return a new
array with a format that matches our path for the route constraint.

This will pull the name of any view ending in .html.erb, including partials, in the app/views/static directory.

The output looks something like this:

["/index",
 "/contact",
 "/success",
 "/privacy",
 "/pricing",
 "/about",
 "/terms"]

After checking all the pages in the static directory, I add /admin, /404, and /500 to the constraint for the other routes outside of my static pages it was having conflicts with.

Now, whenever I add a new page to my static controller, that route will be added to the whitelist of static pages that should not be checked against shortened routes.

Like I mentioned before, I ended up going a different route (see what I did there?) but thought it was a good exercise in over-engineering a solution for the fun of it.

Automating Updates to Twilio Webhook URLs

Cody Norman — Thu, 16 Jun 2022 16:57:50 +0000

This was an interesting challenge I ran into recently and has been the
first time in a while that I've felt that itch to solve a problem just
because I wanted to see if I could do it.

It's been (checks notes) about 7
years since the last time I wrote anything related to ruby and probably 5 years
since I've given any talks or presentations.

I don't think I realized how burnt out I was after a few tumultuous years both
personally and professionally. It was great to feel like I had to solve a
problem just for fun and was even compelled to write about it.

That being said, ya boy is rusty, so bear with me.

Recently, I've been doing some work on a side project that uses Twilio for sending and receiving text messages.
I've always thought of SMS as a really powerful tool to add to any app's arsenal since it leverages something
users already use multiple times per day. In a recent report, Twilio even
reported that 96% of messages are read within three minutes of receipt, and 90% within three seconds!

This post won't go into detail on sending text messages. If you'd like some
more information on how to send text messages, the
twilio-ruby gem has some examples.

Twilio phone numbers have webhooks for both Voice and SMS. These webhooks are
called whenever your Twilio phone number gets an inbound text or phone call.

Incoming message example

Both channels have a fallback URL if the first request to the webhook fails.

Sending Webhook Requests To Your Local Environment

If you've ever developed features from a third-party webhook or service with your local
environment, you probably did it by exposing your local environment to the
internet with something like ngrok.

If you haven't heard of ngrok, it's one of my favorite tools that I've been
integrating into more workflow more and more. You can read more about ngrok
here.

Running Ngrok In Your Procfile

With Rails 7 embracing using foreman and Procfiles for starting and running
services, I've added ngrok as its own process so it's always available.

example Procfile.dev


# Procfile.dev
web: bin/rails server -p $PORT
css: yarn build:css --watch
js: yarn build --reload
worker: bundle exec sidekiq
stripe: stripe listen --forward-to $PORT/webhooks/stripe
ngrok: ngrok http --log=stdout $PORT

Updating Twilio Webhook URLs

With ngrok running, there's now a URL to point your webhooks to.
One option for updating your webhook URLs is grabbing your ngrok URL, going into
the Twilio console and update the webhook URLs for voice and sms.

That was fine for me...for a while.

My main issue with this setup is whenever I restart my services, the ngrok URL
changes and needs to be updated again. That can make for a lot of updates when
starting and stopping your services and errors when forgetting to update the
URLs each time.

Also, if that's something you're not doing very often, it's easy
to forget how to find the section to update the webhook URLs.

That's when I felt the urge to find a way to automate that whenever I needed it.

After some digging in the Twilio documentation, I found some options to set the
webhooks URLs through their ruby gem

A little more digging helped me find an endpoint for ngrok you can query to get
it's public url.

Now we're cooking with gas. With the URL of our local ngrok instance and a way
to programmatically update the webhook URLs, we have the 2 pieces we need.

The next step is to decide how to execute that code when we need updates.
For me, that was a rake task. That gives me a nice repeatable way to update the
URLs whenever I need to.

Rake Task For Updating URLs

Before I start going into more detail, here's the code for the rake task. The
dependencies for the rake task are a running ngrok instance, the twilio-ruby
gem (I'm using v5.67.1),
and if you'd like a pop of color in your terminal to make things easier to spot, the colorize gem

Here is an example of the rake task

namespace :development do
  desc "updates twilio webhook urls to ngrok url"
  task set_twilio_webhooks: :environment do
    return unless Rails.env.development?
    begin
      response = JSON.parse(Net::HTTP.get(URI("http://localhost:4040/api/tunnels")))
      ngrok_url = response.dig("tunnels", 1, "public_url")
    rescue Errno::ECONNREFUSED
      puts "*******NGROK not running*******".yellow
    end
    if ngrok_url
      puts ngrok_url.green
      account_sid = Rails.application.credentials.dig(:twilio)[:account_sid]
      auth_token = Rails.application.credentials.dig(:twilio)[:auth_token]
      development_incoming_phone_number = Rails.application.credentials.dig(:twilio)[:auth_token]
      twilio_client = Twilio::Client.new(account_sid, auth_token)
      twilio_client.api.v2010.accounts(account_sid).incoming_phone_numbers(developemnt_incoming_phone_number)
        .update(sms_url: "#{ngrok_url}/webhooks/twilio",
          sms_fallback_url: "#{ngrok_url}/webhooks/twilio_fallback",
          voice_url: "#{ngrok_url}/webhooks/twilio_voice",
          voice_fallback_url: "#{ngrok_url}/webhooks/twilio_voice_fallback")
      puts "Webhook URLs updated!".green
    rescue Twilio::REST::RestError => e
      puts e
    end
    end
  end
end

The Deets

Now, let's dig a little deeper into what's going on.

Inside the rake task, the first thing I'm doing is making sure we're in the
development environment and returning from the task if we're not. Probably not
required, but I thought it was a nice check to keep from accidentally updating
information outside of our development environment.

After ensuring we're in our dev environment, we need to do is find the URL of the
running ngrok instance. Ngrok has an API endpoint running at port 4040 for the
current tunnels. This endpoint returns an XML response with information about
the current tunnels. There are 2 tunnels returned, and I grab the one with the
name command_line (http) which is the second one returned.

Usually, my ruby HTTP library of choice is HTTParty but I wanted to set this up using
the ruby Net::HTTP lib to keep from introducing another dependency.

After ensuring we're in the development environment, Net::HTTP then makes a
request to the ngrok endpoint and parses the XML response.

Here are some examples of the response it sends back.

Truncated Ngrok response

<tunnelListResource>
  <Tunnels>
  </Tunnels>
  <Tunnels>
  </Tunnels>
  <URI>/api/tunnels</URI>
</tunnelListResource>

Single Tunnel Object

<Tunnels>
  <Name>command_line</Name>
  <URI>/api/tunnels/command_line</URI>
  <PublicURL>https://7be0-72-42-102-194.ngrok.io</PublicURL>
  <Proto>https</Proto>
  <Config>
  <Addr>http://localhost:80</Addr>
  <Inspect>true</Inspect>
  </Config>
  <Metrics>
  <Conns>
  </Conns>
  <HTTP>
  <Count>0</Count>
  <Rate1>0</Rate1>
  <Rate5>0</Rate5>
  <Rate15>0</Rate15>
  <P50>0</P50>
  <P90>0</P90>
  <P95>0</P95>
  <P99>0</P99>
  </HTTP>
  </Metrics>
</Tunnels>

The rake task takes the parsed XML and grabs the 2nd Tunnel returned. Why you
ask?

Honestly, it was most likely the first thing that worked from StackOverflow and I moved
on.

The <PublicURL> for both tunnels seem to always be the same so it's
possible either would work.

If ngrok is not running, we'll get an Errno::ECONNREFUSED error so we can
rescue that error, and print some output in yellow to make it easy to know when
something went wrong

Grab Ngrok Public Url

  begin
    response = JSON.parse(Net::HTTP.get(URI("http://localhost:4040/api/tunnels")))
    ngrok_url = response.dig("tunnels", 1, "public_url")
  rescue Errno::ECONNREFUSED
    puts "*******NGROK not running*******".yellow
  end

Now we have a repeatable and reliable way to grab our ngrok URL whenever it is
running. We also get some colored output to give us a quick visual indicator
that everything looks a-ok.

The next part is to programmatically update the Twilio webhook endpoints with our
public URL for ngrok.

It took quite a bit of digging to find the endpoints needed in the twilio-ruby
gem, but once I found them, it was smooth sailing from there.

Updating with the Twilio API

There are a few things we'll need to update the webhook URLs for Twilio. The
first thing we need is a client with our keys.

To create the Twilio client, you'll need to get your account_sid and
auth_token. You can find both of those by logging into Twilio. While you're
there, be sure to find the service id of your phone number. That's required to
target the correct phone number for the updates.

Twilio Updates

  account_sid = Rails.application.credentials.dig(:twilio)[:account_sid]
  auth_token = Rails.application.credentials.dig(:twilio)[:auth_token]
  development_incoming_phone_number = Rails.application.credentials.dig(:twilio)[:auth_token]
  twilio_client = Twilio::Client.new(account_sid, auth_token)
  twilio_client.api.v2010.accounts(account_sid).incoming_phone_numbers(developemnt_incoming_phone_number)
    .update(sms_url: "#{ngrok_url}/webhooks/twilio",
      sms_fallback_url: "#{ngrok_url}/webhooks/twilio_fallback",
      voice_url: "#{ngrok_url}/webhooks/twilio_voice",
      voice_fallback_url: "#{ngrok_url}/webhooks/twilio_voice_fallback")
  puts "Webhook URLs updated!".green
rescue Twilio::REST::RestError => e
  puts e
end

I included the Twilio client setup in the rake task, but that was mostly to have
an example that would work as-is without additional configuration. I typically
configure my Twilio client in an initializer to keep duplication to a minimum.

example config/initializers/twilio.rb file

Twilio.configure do |config|
  config.account_sid = Rails.application.credentials.dig(:twilio)[:account_sid]
  config.auth_token = Rails.application.credentials.dig(:twilio)[:auth_token]
end

With the client created, we can use the ngrok_url combined with the endpoint
that points to your webhook handler and update our Twilio webhook URLs.

If everything goes as planned, we'll output a success message in green.

If things don't go as planned, we're rescuing Twilio::REST::RestError and
outputting the error.

Now, whenever we need to update our Twilio webhook URLs (or
any other 3rd party service in the future) we can run:


$ bin/rails development:set_twilio_webhooks

Finding a way to automatically run ngrok in my Procfile and have a way to
programmatically grab its public URL was a great find for my workflow. This is
something that can
be expanded to tons of other services you may like to work with in your development
environment.

Let me know any other services you automate!