DEV Community: Adam Rogers

I’m on Team Good Commit Messages

Adam Rogers — Sat, 04 Feb 2023 17:10:22 +0000

There has been some debate on the value of good commit messages, and if we need them at all.

I’m on team good commit messages for a few reasons.

Good commit messages act as a sanity check for my changes

I cannot count the number of times I’ve been writing out a commit message, thought to myself “this is a stupid way to do this”, binned it all off and started again.

Good commit messages help me prove to myself that I know what I’m doing

If you understand something, you can explain it simply. If you can’t explain it simply, you don’t understand it. A good commit message is a simple explanation of the change, and I like to try to understand the changes I’m making to the code.

Good commit messages are integral to Git

Git is based on commits and commits have messages. Using Git effectively means working with commits, and commit messages the key to working with Git. It isn’t an intuitive tool, but it is a powerful one. Learning to use it efficiently is worth while, and good commit messages make working with Git easier.

Good commit messages give context for code

I have a plug-in for my text editor that makes it easy to view the commit associated with a given line of code, and the commit messages of the lines around it. The unfortunately named git blame command can often give more context than a comment in the code that is more likely to be out of date, or a link to an issue or other URL that is similarly tired or even expired.

In my experience, the value of good commit messages really becomes apparent when everyone on the team takes the time to write them. This is easiest to achieve if it’s just you, but wonderful to see when the whole team “gets it”.

This isn’t to say that I always write good commit messages. I don’t. Sometimes I have no idea what I’m doing, or I’m rushing a hot-fix into production, or a just can’t be bothered. I’m not a martyr.

But, given the time and space, I’m on Team Good Commit Messages.

Updating Rails applications with vimdiff

Adam Rogers — Wed, 09 Nov 2022 14:30:42 +0000

This is probably one of those things everyone knows about, but I only just figured it out, so here it is for the next person (likely me, six months from now).

Ever run bin/rails app:update, been greeted with this:



   identical  config/boot.rb
       exist  config
    conflict  config/application.rb
Overwrite /home/rodreegez/Projects/my-app/config/application.rb? (enter "h" for help) [Ynaqdhm]

...and wondered what to do?

Y will overwrite your existing file with the new one. d will show you the diff between your existing file and the new one. But what if you want the new stuff but also keep your changes?

vimdiff to the rescue.

The option we're looking for is m for merge. To use it, we have to set the THOR_MERGE environment variable and tell it which tool to use to show the diff.

Let's cancel out of the update process we just started and try again, this time with vimdiff:



THOR_MERGE=vimdiff bin/rails app:update

This time, when we press the m key, vimdiff will spring into life. It might look something like this:

In the diff we can see that I have made some changes to application.rb (on the right - my additions are highlighted in green) and I'd like to keep those.

The goal of this exercise is to make both of these files the same. We do that by :diffgeting and :difputing changes - diffget "gets" the corresponding change from the other file, while diffput "puts" the change from this file to the other one.

Many changes will be of little consequence - strings being changed from single to double quotes, for example - but some will require a bit of thought.

Both commands work from the perspective of the window we're currently in. I tend to think it's easier to think about if I ctrl-w over to "my" version of the file and get and put changes from there.

Every time we diffget or diffput a change, that change will disappear and we're free to cursor down to the next highlighted change. Once there are no more changes left we can :wqa which will save and quit everything, and our update script will move on to the next file.

Rinse and repeat with all the files that bin/rails app:update touches and, by the end of it, you'll have a nice and up-to-date Rails application with all your customisations, personalisations and preferences still intact.

Don't forget to configure any new framework defaults and check for any further changes you might need to make. And, of course, make sure your tests still pass.

GeoJSON, PostGIS and the Right-Hand Rule

Adam Rogers — Wed, 24 Aug 2022 08:21:00 +0000

GeoJSON expects Polygon and MultiPolygon features are required to be "right-hand wound". I understand this to mean that the order of the coordinates that make up the polygon should move in a specific direction.

Unsurprisingly, exactly which specific direction is somewhat ambiguous. The GeoJSON spec says:

Polygon rings MUST follow the right-hand rule for orientation (counterclockwise external rings, clockwise internal rings).

So, obviously, a simple 12-point Polygon in the shape of a circle and following the right-hand rule starts at, say, noon and the next point will be at, say, position 11 on the clock face.

In my book the primary direction of travel in that case above is, well, left. So "right-hand rule" means "goes left".

So why are my polygons failing on GeoJSONlint.com? I produce my polygons by exporting them from my Postgres database using PostGIS.

PostGIS has a function to force polygons to follow the right-hand rule. The documentation for ST_ForceRHR states:

Forces the orientation of the vertices in a polygon to follow a Right-Hand-Rule, in which the area that is bounded by the polygon is to the right of the boundary. In particular, the exterior ring is orientated in a clockwise direction and the interior rings in a counter-clockwise direction.

So "right-hand rule" in PostGIS-land means... exactly the opposite of the same rule in GeoJSON land. And also matches my intuitive interpretation of "right-hand rule" as applied to polygons.

To create correctly "wound" polygons in GeoJSON when exported from PostGIS, the solution is (obviously) to apply ST_ForcePolygonCCW when exporting polygons as JSON from PostGIS.

References

This Stack Overflow comment was key to understanding all of the above: https://gis.stackexchange.com/questions/306567/postgis-geojson-query-not-validating#comment496061_306571

Connect a local Rails app to an RDS Instance via an SSH tunnel through a bastion server

Adam Rogers — Mon, 13 Dec 2021 08:50:33 +0000

Editor's note: this is supposed to be a bit tongue-in-cheek. We've all felt like this before!

Right, here's what we're doing; local Rails app connects to a local Postgres instance in the usual way. Fine. But because reasons it also connects to another database that is massive and a total Pain In The Ass (PITA).

Because "The Man" cares about security, the only way to access that second RDS instance is through a "bastion" server, which makes this a 10x PITA. Here's probably the worst guide in the world to setting up the sort of configuration we're working with.

I can't believe I have to do this. I hate computers.

Fine.

Connect to the bastion server

The first thing we have to be able to do is to access the bastion server from our local machine. Let's assume you know how to do this. You basically want to run something like ssh user@ip where user is the username of the account you'll be accessing the machine as and ip address is the IP address of the stupid computer getting in the way of you and your multi-database dreams. You'll want to add your ssh keys to the box and stuff. Figure it out.

Then we have to connect from the freakin' bastion server to the RDS instance living inside it's ivory tower with no bloody internet access. Urgh.

On the bastion server we should be able to run something like psql -h "<a.very.long.hostname.us-east-1.rds.amazonaws.com>" name_of_database username. Fill in the blanks, smash that enter key and probably enter a password. You know the password, right? Right.

Great. We can access the database. This is 10x engineering right here - accessing a sodding database. Welcome to the bleeding edge.

Now we have our computer talking to the bastion server and the bastion server talking to the RDS. Wonderful. I'm so pleased. Now all we gotta do is get this box on our desk to talk to the RDS instance through the bastion server. Joy.

Tunnel from the bastion server to the RDS instance

This is so annoying.

What we want is to forward a port on our local machine to a port on the bastion server, which in turn forwards traffic on that port through to the appropriate port on the RDS server, adding absolutely nothing of value on the way except "security".

Here's how we do that. ssh has a -L flag that does... something. Basically you give it the local port you want the connection for forward, the host you want it forwarded to, and the port on that host that the traffic should end up on. Or something like that. I don't know. I didn't really read the man page.

For us, that makes the whole thing look something like this:

ssh -L 5433:a.very.long.hostname.us-east-1.rds.amazonaws.com:5432 username@bastion-ip-address

Actually using it

That's a bit of a mouthful to type out every time, so here's what we're going to do; we'll use the ~/.ssh/config file to set configure an ssh connection and specify the tunnel in there so that every time we ssh into the box the tunnel will be set up and our connection to the remote database will Just Work™ (aye, right).

Here's what our ~/.ssh/config file is going to look like:

# ~/.ssh/config

Host stupid-computer
  HostName <bastion IP address goes here>
  User <bastion username goes here>
  IdentityFile ~/.ssh/id_rsa
  LocalForward 5433 <a.very.long.hostname.us-east-1.rds.amazonaws.com>:5432

(The LocalForward bit isn't exactly the same as when we used the -L flag. It's LocalForward <local port> <remote database uri>:<remote port> with a space between the <local port> and the rest of it.)

This tells ssh that when we run ssh -Nf stupid-computer, it should connect to the bastion server and forward any traffic on local port 5433 to remote port 5432 on the RDS instance.

-N tells ssh not to execute any remote commands (just forward ports) and -f tells ssh to fork into the background. This has the effect of you now not knowing what the h*ck ssh is doing anymore, but we'll do something about that later.

For now you can ps aux | grep ssh to find the process id and then kill <process id> when you want to stop the tunnel.

Configuring Rails

Now we get to hook our Rails app to the database. Again, this is forefront of technology right here. Connecting a CRUD app to a database. Doesn't get any more exciting than this.

# config/database.yml

remmote_db:
    database: name_of_the_actual_db
    username: db_username
    password: db_password
    host: localhost # may need to be 127.0.0.1 if using mysql
    port: 5433

Or if you're not an animal and prefer to connect to the database via a URL:

# config/database.yml

remote_db:
    url: <%= ENV['REMOTE_DB_URL'] %>

and in yr .env file (or however you go about managing that sort of thing):

# .env

REMOTE_DB_URL: postgres://<db_username>:<uri_encoded_db_password>@localhost:4533/<name_of_the_actual_database>

Hooking it all up

Run ssh stupid-computer and then boot yr Rails app. Assuming you have an Active Record model that looks a bit like this:

class Whatever < ApplicationRecord
  establish_connection :remote_db
  self.table_name = "whyyy"
end

You should be able to ./bin/rails c and Whatever.count should return the number of rows in the whyyy table in the public schema on remote_db.

And that's about it. You need to have the ssh connection to the bastion server up and running in order for your model to be able to access that database.

Wrangling that ssh process

This is all fine and it works but we have to have the ssh process running and we're definitely going to forget to do that and then it'll all break and we'll wonder why and why oh why do we have to do these things?

The main problem is that output of ssh stupid-computer doesn't really give us any feedback, so we don't know if that's up and running.

We can fix that with some more ~/.ssh/config and a couple shell aliases (shamelessly stolen from this post).

We're gonna tell ssh to connect and create a control socket (whatever that is) for our connection by passing it the -M flag and updating our config.

To make our lives easier, we'll create some aliases in .bashrc (note the addition of the M to the command we used above).

alias sc-up="ssh -NfM stupid-computer"
alias sc-status="ssh -TO check stupid-computer"
alias sc-down="ssh -TO exit stupid-computer"

And we'll specify where we want the control socket to live by adding a line to our ~/.ssh/config:

# ~/.ssh/config

Host stupid-computer
  HostName <bastion IP address goes here>
  User <bastion username goes here>
  IdentityFile ~/.ssh/id_rsa
  LocalForward 5433 <a.very.long.hostname.us-east-1.rds.amazonaws.com:5432
  ControlPath ~/.ssh/stupid-computer.ctl

Now when we want to start the ssh tunnel we can simply sc-up. When it blows up we can run sc-status, see we didn't start the tunnel yet and run sc-up, and we can kill it all off when we have given up on life with a simple sc-down.

Conclusion

Computers are rubbish. "The cloud" is just somebody else's computer, therefore is also rubbish. The struggle is real.

Now if you'll excuse me I have some actual work to do.

References:

https://myopswork.com/ssh-tunnel-for-rds-via-bastion-host-6659a48edc
https://stackoverflow.com/questions/26774264/how-do-i-configure-rails-for-password-less-access-to-remote-database
https://dev.to/thebenforce/connect-to-a-postgresql-server-via-ssh-tunnel-832
https://aws.amazon.com/premiumsupport/knowledge-center/rds-connect-using-bastion-host-linux/
https://codingsight.com/connecting-a-bastion-server-to-a-postgresql-server-via-secure-shell-tunnel/
https://mpharrigan.com/2016/05/17/background-ssh.html

Rails System Tests with Headless Chrome on Heroku CI

Adam Rogers — Fri, 04 Jun 2021 15:20:41 +0000

Sometimes your Rails apps just need to execute JavaScript in a real browser. But you also want to run tests on CI, which typically doesn't have a UI. So you need to configure Google Chrome to run headless, i.e. without popping up a browser window (but still exercising your app as if it were).

Urgh.

Having just worked through this now, here's what worked for us.

Prerequisites

Install chromedriver.

Configure test environment

In your app.json, add the chromedriver and google-chrome buildpacks, as well as the usual ruby one. Our app.json ended up looking like this:

{
  "environments": {
    "test": {
     "buildpacks": [
        { "url": "https://github.com/heroku/heroku-buildpack-ruby" },
        { "url": "https://github.com/heroku/heroku-buildpack-chromedriver" },
        { "url": "https://github.com/heroku/heroku-buildpack-google-chrome" }
      ],
      "addons": [
        "heroku-postgresql"
      ],
      "scripts": {
        "test-setup": "bin/rails assets:precompile",
        "test": "bin/rspec"
      }
    }
  }
}

Also, set any environment variables you might need in your test env via the Heroku UI. You can set non-sensitive stuff in app.json if you want, though.

Configure Capybara with a headless driver

# spec/rails_helper.rb

chrome_bin = ENV.fetch('GOOGLE_CHROME_SHIM', nil)

chrome_opts = chrome_bin ? { "chromeOptions" => { "binary" => chrome_bin } } : {}

opts = Selenium::WebDriver::Chrome::Options.new

chrome_args = %w[--headless --no-sandbox --disable-gpu]
chrome_args.each { |arg| opts.add_argument(arg)  }

Capybara.register_driver :headless do |app|
  Capybara::Selenium::Driver.new(
     app,
     browser: :chrome,
     options: opts,
     desired_capabilities: Selenium::WebDriver::Remote::Capabilities.chrome(chrome_opts)
  )
end

Capybara.javascript_driver = :headless

...and that's about it.

The bit that we found we had to do that was different from other posts of this nature we found was setting Selenium::WebDriver::Chrome::Options.new. Without this, everything we tried to tell Chrome to be headless was ignored.

If you're reading this, I hope it helps!

References

Password management on the command line

Adam Rogers — Sat, 27 Feb 2021 21:52:24 +0000

pass is a command line password manager for Linux that embodies the UNIX philosophy. It's like 1Password, but it lives in your terminal. This comes with some drawbacks, certainly, but is also very powerful.

Here we'll get pass installed and then explore a couple reasons why you might like a Command Line Interface (CLI) to your password manager.

Installation

I'm using Ubuntu 18.04 (I know), but these instructions should work for any apt based distribution, and should be easily adaptable to your *nix of choice.

First, we'll make sure we have pass itself, gpg which comes with Ubuntu by default and will be used to encrypt our passwords, and also xclip, which we'll use to copy passwords to the system clipboard.

$ sudo apt install pass gpg xclip

Getting started

Before we can use pass, we'll need a generate a GPG keypair. To do that, we'll kick off the keypair generation wizard like so:

$ gpg --full-generate-key

Choose the default kind of key (RSA), and default keysize (3072). Set the expiry to 0 (doesn't expire). Confirm these options with y.

Next, it will ask you to create a user ID to associate with this key. First you'll be asked your name - I can't help you answer that. Next, you'll be asked for your email address - this is kinda like the "username" for your GPG key. Finally you'll be asked if you'd like to supply a comment - you can skip this by just hitting enter or give your job title if you're very important. Choose O to confirm your information.

Next, you'll be asked to enter a passphrase. This should be 8+ characters long. You'll need to remember it, but it should be secure. Think of something appropriate, enter it, then enter it again to confirm.

GPG will now generate a random key. Wiggle your mouse around and mash the keyboard to generate entropy while it does this. I have no idea how it does this or if this is a joke, but this is security we're talking about here, so get mashing.

Initialize `pass`:

Now we get to start using pass. First we create the initial password store, passing it our GPG key ID (which is the email address we used above).

$ pass init the-email-address-you-used-for@your-gpg.key

Using pass

pass stores passwords in a hierarchical directory structure. This structure is somewhat arbitrary, but basically follows the structure service-name/username. If you have multiple accounts on the same service, you might nest those accounts under a parent folder, e.g. aws/development/adam@example.com and aws/production/adam@example.com - each of these paths will lead to a different password.

Adding passwords

Add a new service and set the password (you'll be asked to type the password and then type again to confirm):

$ pass insert test-service/adam@example.com

If you don't know the password already (i.e. you're signing up for a new service), you can generate a random password instead of making one up yourself (where 16 is the length you want the password to be):

$ pass generate test-service/adam@example.com 16

Viewing passwords

To view the password you just generated, or indeed any password in the vault, simply pass pass the path to the value you'd like to retrieve (you'll be asked to enter your GPG password from earlier):

pass test-service/adam@example.com

To view that generated password and copy it to the system clipboard):

pass -c test-service/adam@example.com

To list all available accounts (but not their passwords):

$ pass ls

Removing passwords

To remove a password (you'll be asked to confirm deletion):

$ pass rm test-service/adam@example.com

Remove all passwords in a given directory (you'll be asked to confirm deletion):

$ pass rm -r test-service

...and because pass follows the UNIX philosophy, you can find instructions for loads more knobs and whistles with man pass.

Really using `pass`

So far, so good. You're a l33t hacker and now use a CLI password manager. Your passwords are stored locally and you can only get to them from your terminal. Congratulations, you just made your life just a little more difficult.

Or did you?

Here's the thing. pass doesn't have to store passwords. It can store any string at the end of those paths. And those paths don't have to map one-to-one with a username and password combination.

Use `pass` to store secure API keys

One of the most interesting uses of a CLI password manager is that it can integrate with your shell. That means if we can use pass in our bash scripts. This gets really useful when we combine that with e.g. loading ENV VARs for our projects.

Say we had the following passwords set:

$ pass ls
Password Store
└── aws
    ├── secret_key
    └── access_key

We could then set these in a .env file, and use dotenv to read those values when booting our application (as described elsewhere):

AWS_SECRET_KEY: $(pass aws/secret_key)
AWS_ACCESS_KEY: $(pass aws/access_key)

When we boot our app, the $() syntax will get expanded out, pass will be called, and our secret_key and access_key will be interpolated in place. Very cool. We can check this file into source control and no one will know what our AWS credentials actually are.

If we have multiple credentials for the same service, we could separate them out however we like - pass doesn't care. We could have aws/staging/access_key and aws/production/access_key (maybe don't store prod access keys locally though), or aws/staging/client_a/access_key and aws/staging/client_b/access_key. Whatever works for you.

Conclusion

We've only just scratched the surface of what one can do when your credentials are managed on the command line. Next up I'd like to figure out how to get credentials shared between machines / teams through git, ensuring only those with approved GPG keys can access the secrets held within. And you'll be surprised at how flexible having your passwords can be in practice, once you get your head around it.

I still use a regular password manager for regular passwords, but for credentials like API keys and tokens, pass is quickly becoming a handy tool.

References

Rails FormBuilders with TailwindCSS

Adam Rogers — Mon, 15 Feb 2021 22:11:03 +0000

TailwindCSS is all about adding lots of utility classes to your markup to style elements directly. The problem with that, of course, is that you end up adding lots of utility classes to your markup. Like, a lot of classes. Dozens, sometimes, by the time you've accounted for the various break points and perhaps dark mode and what have you.

This can become tiresome, especially in the case of forms, where each element of the form may have all those classes applied and these classes need to be repeated for a dozen of elements in the form. Nobody has got time for that, I'm sure.

You want all the fields in your form to be consistent with each other, but you also want all the forms on your site to be consistent with each other, too. If you want to change how a text_field tag looks you want to make that change in one place and see the change reflected everywhere. DRY, and all that.

Tailwind encourages us to extract components in our views as a way to reduce duplication. In Rails we also have the concept of partials that we can use to share view code between different contexts. If we have a form, we might extract it into a partial and use the same form in the new and edit views.

That's cool, but we still have to duplicate all the utility classes on every element in that form, and we can't share elements between forms for different things.

Not very DRY.

FormBuilders

But Rails has been doing this stuff for years, and obviously there is a super easy and well thought out solution for solving this exact problem: FormBuilders.

We can create our own FormBuilder, supplement the default behaviour with our TailwindCSS utility classes, and then whenever we write f.text_field :whatever we'll get a nicely styled text field that looks the same as every other text field on our site.

Here's how to do that (presuming you already have TailwindCSS in your Rails application, which is beyond the scope of this article).

Install the Tailwind forms plugin

To get us off to a good start, install the Tailwind forms plugin. This'll give us some nice forms out of the box, and allow us to style things further.

Create a form builder

I like to create a folder in app to keep form builders in. Files in subfolders of app are automatically loaded, and if we decide to add more form builders later then there is a sensible place to put them.

The basic form builder looks like this:

  class TailwindFormBuilder < ActionView::Helpers::FormBuilder
    def text_field(method, opts={})
      default_opts = { class: "mt-1 block w-full rounded-md border-gray-300 shadow-sm focus:border-indigo-300 focus:ring focus:ring-indigo-200 focus:ring-opacity-50 #{'border-2 border-red-500' if @object.errors.any?}" }
      merged_opts = default_opts.merge(opts)
      super(method, merged_opts)
    end
  end

We create a new class that inherits from ActionView::Helpers::FormBuilder and then go ahead and override the text_field method it provides. In our version, we're going to slam a bunch of Tailwind class names into the options hash we're given by default. This allows us to set different classes on a case by case basis but it's a fairly crude implementation so if we want to override one class we have to supply all the classes we want to see applied to the final element. Finally, we super the original method we were passed and our freshly mangled opts hash up to the original text_field method we're overriding and let it do its thing.

A more sophisticated solution is left as an exercise to the reader.

Use the FormBuilder

We can then use the form builder very simply by passing it to form_with:

  <main class="p-12 mx-auto text-gray-800 max-w-7xl">
    <%= form_with model: @widget, builder: TailwindFormBuilder, class: "mt-6" do |f| %>
      <%= f.label :reference %>
      <%= f.text_field :reference, placeholder: "Your reference" %>
      <div class="pt-5">
        <%= f.submit "Begin" %>
      </div>
    <% end %>
  </main>

When we render the form and take a look at our text field, we see all our wonderful TailwindCSS classes!

<input class="mt-1 block w-full rounded-md border-gray-300 shadow-sm focus:border-indiogo-300 focus:ring focus:ring-indiogo-200 focus:ring-opacity-50 " placeholder="Your reference" type="text" name="widget[reference]" id="widget_reference">

We can rinse and repeat this process for any and every Rails form tag we want to apply sensible default TailwindCSS styles to.

Making TailwindFormBuilder the default

Making our form builder the default is easy, too. In our ApplicatioHelper, we can set it as the default like this:

ActionView::Base.default_form_builder = TailwindFormBuilder

We can then remove builder: TailwindFormBuilder from our call to form_with in the view, and all forms will use our nicely styled text_fields by default.

Conclusion

This is obviously a super simple example of what you can do with FormBuilders in Rails and how they can help tuck those CSS utility classes out of the way. We can do some neat things like making the border of the text_area red if the object has errors. We can even start cracking out content_tags to create chunks of HTML that include labels and inline error messages for each element. So far I'm still trying to find the balance between "everything is in the form builder" and "nothing is in the form builder". You'll notice above that the form is still managing a bit of it's own spacing, which may or may not be worth moving into the TailwindFormBuilder, but I so far I've kept label seperate from text_field, for example, and I think I'm happy with that.

I'm currently experimenting with appending a ul of error messages to each element in the form as a way of displaying nicely styled inline error messages to the user. This seems to be a bit of a reach, at first, because the markup for the errors don't appear in the form itself. But because they are applied by the form builder, they are applied consistently and to every text_field that the form is used for. That's just how errors are dealt with in this kind of form. I quite like that (so far).

FormBuilders are a great way to tuck utility CSS classes away in Rails applications.

Refernces

Bridgetown and Environment Variables

Adam Rogers — Mon, 07 Dec 2020 19:57:11 +0000

This afternoon I had a bit of a time getting environment variables working in my first real stab at a Bridgetown site. I'm not sure I've figured out the best way to do this, but here's what I came up with.

Background

It's worth pointing out a few related background details:

I am a curmudgeon and I do not like change, so I switched the templating in my site to erb, which I believe is a relatively new feature of Bridgetown, but a relatively old feature of Rails.
I want to be able to set env vars locally, keep them out of my git repo, and set different ones in production in whatever way is apropriate for whatever production is.
I don't want my application code to care about what environment it's running in.

dotenv

I usually make Rails applications, and with Rails applications I usually use dotenv. This allows me to have a .env file that gets loaded when my Rails app boots. In Rails-land, we can use dotenv-rails which will hook into the application boot process and bootstrap our environment variables for us. This mostly works fine, but you can mess with it if you need to. Yay for Rails, I guess.

Bridgetown is not Rails

How, then, to hook the loading of env vars into the build process? After a bit of messing about and a nudge from the nice people on the Bridgetown discord server I discovered that one can write "plugins" for Bridgetown. Getting dotenv to play nice with bridgetown was as simple as bundle add dotenv and then creating the file /plugins/env.rb that required the gem:

# /plugins/env.rb
require 'dotenv/load'

Setting Env Vars

With dotenv being required, any key=value pairs we put in a file called .env located at the route of the project will be loaded into the environment when Bridgetown builds.

# .env
MY_API_KEY=ABC123
SOME_OTHER_KEY=DEF246

You're not limited to .env files, either. dotenv allows you to set key-value pairs in a whole bunch of different files, and will load each in an order of precidence.

Keeping API keys out of git

Obviously the whole point of this exercise is to keep our precious API keys out of git. To do this, I like to add .env to my .gitignore, duplicate the file and rename it to .env.example and then replace all the keys in .env.example with nonsense. I do this on basically every project so when I see a .example file I know I need to copy it to the real file, but you may want to add a reminder to your README or similar.

Pro tip: do this before you commit any of your dotenv stuff to git, else you'll have to look up how to make git forget about the original .env file and who's got time for that kind of nonsense?

Reading Env Vars

Now that we have Env Vars being set and loaded, we can read them in our views. As mentioned I'm using erb, so that should look pretty familiar to anyone who's done a little Rails:

<%# /src/_layouts/home.html.erb %>
<%= ENV['MY_API_KEY'] %>

Why did you do this?

Static sites generally and Bridgetown particularly allow us to build "Jamstack" sites that leverage third-party APIs from the client side, and eschew a server entierly. In order to access an API though, you'll typically need to access an API key. Think Stripe API keys, Google Analytics API keys, Mapbox API keys... anything like that.

Bridgetown does have a mechanism, two in fact, for loading arbitrary key-value pairs at build time, but neither felt like a particularly suitable place to put API keys.

config.bridgetown.yml exists and you could add keys there, however this is the global config for Bridgetown, which means you would find it trickier to do the whole .example trick to keep your keys out of git, and if you did you'd have to copy your whole config across which seems like it'd be just itching to get out of sync.

You can also add keys to the site.data object in Bridgetown by creating e.g. /src/_data/env.yml. This would make our keys available through e.g. site.data.env.MY_API_KEY, but that felt like abusing a function meant for something else.

At the end of the day, I only really need to dump the right value for a given key onto the page interpolated into some JS snippit or other, and Env Vars seem like the right place to do that.

How do you load API keys into your Bridgetown site?

Re: Learning to code

Adam Rogers — Wed, 01 Apr 2020 19:31:22 +0000

My dear brother emailed me asking how he could use all this free time he's got in "lockdown" to build an app he'd had an idea for. I'm sure he won't be the only person thinking along similar lines about now. Here's the response I wrote back:

Yo.

Programming, eh?

I am all for encouraging anyone to pick up programming, but I feel like we might need to adjust your expectations a little bit. What you're hoping to make is a pretty complex beast (there are more than a few moving parts) and I'm concerned that you'll be disheartened if you try to bite all that off at once. That Personal Record Club app I built took me like a month and I've been doing this shit for 10 years. I should be a lot better at it after 10 years and I'm not, and I'm certainly not suggesting it'll take 10 years before you could attempt to build your idea, but my point still stands - I didn't just knock it out overnight (despite appearances).

I think the first thing you (or anyone else) should learn to build is a blog. You already understand the domain of blogging (on the homepage there is a list of post titles in reverse chronological order, you click on a title and it takes you to the full article) and you can make it progressively more complex as you do (add a user to author the posts, add many users, show the word count for each post, convert that word count into an estimated reading time, add the ability to upload pictures, add the ability to search an API for recommended pictures, etc. etc. etc.) so you can focus on learning the programming stuff incrementally without having to think about all the edge-cases in the domain as well.

Of course nobody ever wants to build a blog, they want to build the whizzy app they thought up, so no one ever takes this advice. But it's good advice, David. Start with a blog.

Demoralised? Great. Get used to it. Programming sucks.

What stack you build you blog in is up to you. I will be able to help you if you choose Rails, but Rails isn't the new hotness it once was. All the cool kids are learning JavaScript. Obviously all the cool kids are wrong, so choose Rails. Or JavaScript, it's really up to you.

There are loads of resources to learn how to program. Agile Web Development with Rails is the book I used way back when (you build a shopping cart app - they should've taken my advice) but there are loads of online schools for this and that as well.

Codecademy has been around forever and their webdev course looks like it goes down the JS-heavy route in a pretty sane way. I've seen a lot of people talk about The Odin Project for learning Rails these days. I'm sure you can find similar resources for Python (which is basically Ruby without a few keywords but has some awesome data science libraries so all the machine learning and AI kids love it), or any one of a number of other languages.

All I would say is pick a language and stick with it. Try and pick something that is a bit framework-agnostic. At the end of the day, Rails is just a bunch of Ruby, so I think that counts, and while you could say the same for any of the JavaScript frameworks, I think you'll get more mileage out of learning JavaScript at least a bit before diving into something like React, Vue or Angular (do people still use Angular?) which all require a bunch of tooling to get going with and you'll spend half your time learning configuration and not-programming stuff. Which you'll no doubt have to do at some point but maybe not right off the bat, eh?

Hope that helps. I think my main point is that I can help you with Rails but everyone will tell you to do JavaScript. They are probably right, but they should also get off my lawn.

Spoke to mom the other day. Hope the rash is better.

Cheers,

Adam

Immutable ActiveRecord Models

Adam Rogers — Mon, 10 Feb 2020 12:28:54 +0000

I've just implemented something that I think is pretty neat in DeskBeers - an immutable ActiveRecord model.

Why?

Why would you want an immutable ActiveRecord model? In our case, the model in question was Subscription. Subscription, in the DeskBeers sense, holds on to the products a customer wants, and the frequency with which a customer wants them. A customer can update the products in their Subscription and the frequency with which they receive them any time, for example this week you might not want your usual order of wine, or you might have a bit of a backlog so might choose to dial deliveries down to being fortnightly for a while. That's totally cool, we got you.

From the information we hold in a customer's Subscription, we then create Order records at the appropriate time. Because a Subscription can change but an Order (once processed, at least) shouldn't, we have to take a copy of the products requested by the Subscription and duplicate that onto the Order. This way, when we look back at past orders, they retain the information of what was sent to the customer and when, regardless of changes in the parent Subscription.

By making Subscription immutable, we get rid of the requirement for the order to know what the customer requested. When the customer makes a change to their Subscription, we create a new Subscription for them, based off the old Subscription, with the required changes. The Order records that pertain to the old Subscription still hang onto the old Subscription, and new ones take their information from the new Subscription.

This allows us to massively simplify Order and make it way more flexible - anything can be added to an Order (anything skuable, that is) and we can add new LineItems to Order way more easily. I'm going off-track a bit here and, not wanting to get into the weeds, let's just say this is a huge win for the flexibility of the DeskBeers platform, honest.

A nice side effect of this change is that, if a customer so wanted to, they can roll back a change made to Subscription as we now maintain a history of the model, which is cool. Or they could treat these different versions as templates, e.g. have one with wine and one without, and alternate between these presets as they saw fit. If they wanted to. Which I'm not sure is a requirement anyone has actually asked for. But it's basically a free feature, perhaps it'll come in useful.

How?

Largely thanks to this post on the Strikingly blog, I created a module like this:

module ActiveRecord
  module Immutable
    class UpdateImmutableException < Exception
      def initialize
        super("Immutable model can't be updated")
      end
    end

    extend ActiveSupport::Concern

    included do
      before_update :prevent_update
    end

    private

    def prevent_update
      raise UpdateImmutableException.new
    end
  end
end

and included it into Subscription like this:

class Subscription < ApplicationRecord
  include ActiveRecord::Immutable
  # ... code from what is definitely a God model omitted
end

This caused hundreds of test failures. Working through them has proved to be super-interesting, and I've learned a lot about our codebase in fixing them. Making Subscription immutable really highlighted where Subscription and Order were tightly coupled, and fixing the tests has lead me to what I believe will be a way more flexible and ultimately easier to understand codebase.

Do you have any models in your Rails app that could potentially benefit from being immutable?

Creating and updating users with omniauth

Adam Rogers — Fri, 03 Jan 2020 21:57:07 +0000

There are a number of tutorials, including the omniauth overview, that suggest having a method along the following lines:

def self.from_omniauth(auth)
  where(provider: auth.provider, uid: auth.uid).first_or_create do |user|
    user.email = auth.info.email
    user.password = Devise.friendly_token[0, 20]
    user.name = auth.info.name   # assuming the user model has a name
    user.image = auth.info.image # assuming the user model has an image
  end
end

That first_or_create will try to find a user by the auth.provider (e.g. twitter) and auth.uid (e.g. rodreegez) and create one with those attributes if one isn't found. What it doesn't do is update a found user if the user is found.

So, in the example above if, when an existing user signs in with Twitter having updated their name, our app won't get the new name of the user.

Instead, we should do something like this to ensure we are updating our user information when the user logs in:

  def self.from_omniauth(auth)
    user = find_or_initialize_by(provider: auth.provider, uid: auth.uid)
    user.email = auth.info.email
    user.password = Devise.friendly_token[0, 20]
    user.name = auth.info.name   # assuming the user model has a name
    user.image = auth.info.image # assuming the user model has an image

    user.save
    user
  end

That find_or_initialize_by behaves more like we expect it in this situation. Or at least, it behaves in a way that is more appropriate to the task we have at hand - it either finds an existing record or initializes a new one, but doesn't try to save the record. We are then able to set the attributes of the found-or-initialized user, save it, and return the saved-or-updated user to the caller.

Hope that helps.

🍻

Switching to Zeitwerk

Adam Rogers — Mon, 30 Dec 2019 08:46:58 +0000

I’ve just bumped DeskBeers up to Rails 6.0. One of the new things in this release of Rails is the new way files get autoloaded. zeitwerk is very cool, and you should check it out. I won’t go into the pros, cons, whys and wherefores of the change in this post — many folks smarter than I have attested to those — however I do want to bring up a gotcha and solution that bit me while rolling out the change.

Zeitwerk loads constants based on paths

Again, without wanting to go too much into how stuff works, you need to know that, basically, Zeitwerk sees the path app/models/user.rb and from that knows to autoload the constant (class) User. Which is fine, except when it isn’t.

For example, DeskBeers charges people VAT, and is old enough to have lived through a change in the applicable VAT rate. So, for better or worse, we now have a model in a file called app/models/vat.rb. Zeitwerk sees this and tries to autoload the constant Vat. But because the concept being described is an acronym, the constant (class name) defined in the file isn’t Vat, it’s VAT.

After switching to zeitwerk, we get the error:

Zeitwerk::NameError (expected file /.../app/models/vat.rb to define constant Vat, 
but didn't)

That’s the error, but it’s easy enough to fix.

Make sure all your acronyms are properly inflected

Rails provides a simple way for you to let it know about acronyms used in your codebase. By adding them to config/initializers/inflections.rb, Zeitwerk will pick up any acronyms like this and load VAT instead of Vat. DeskBeers (now) defines the following:

ActiveSupport::Inflector.inflections do |inflect|
  inflect.acronym 'VAT'
  inflect.acronym 'CSV'
  inflect.acronym 'PDF'
end

Once you’ve done this, you’ll need to grep your project for all occurrences of Vat and replace them with VAT. Note: this applies to partial class names, too, e.g. VatCalculator will need to be updated to VATCalculator and of course every reference to these classes need to be updated as well — time to break out that find and replace in project function.

I think this is a Good Thing, as it’s made us be deliberate about naming things. For instance, in our case, VAT was always inflected, but PDF wasn’t and CSV had clearly been inflected at some point before today but after it was first used, so we had a mix of CSVWhatever and CsvWhatever classes. After fixing this issue, all our acronyms are now properly inflected and consistently referenced across our codebase.

You can also create an initializer for Zeitwerk itself, and for some files that might be appropriate, but in the case where we’re using acronyms, using the Rails inflections initializer, of which Zeitwerk is aware out of the box, forces us to properly name classes so that they can be loaded, without the need for an extra initializer.

Finding problem classes

The best way to make sure Zeitwerk can load all your classes is to have Zeitwerk load all your classes. This won’t happen by default in development mode in Rails, but you can force it by booting up a console and running:

Zeitwerk::Loader.eager_load_all

If you have any sketchy classes, you’ll get an error like:

Zeitwerk::NameError (expected file /.../app/models/csv_whatever.rb to define constant CSVWhatever, 
but didn't)

Hope that helps.

🍻

DEV Community: Adam Rogers

I’m on Team Good Commit Messages

Good commit messages act as a sanity check for my changes

Good commit messages help me prove to myself that I know what I’m doing

Good commit messages are integral to Git

Good commit messages give context for code

Updating Rails applications with vimdiff

GeoJSON, PostGIS and the Right-Hand Rule

References

Connect a local Rails app to an RDS Instance via an SSH tunnel through a bastion server

Connect to the bastion server

Tunnel from the bastion server to the RDS instance

Actually using it

Configuring Rails

Hooking it all up

Wrangling that ssh process

Conclusion

References:

Rails System Tests with Headless Chrome on Heroku CI

Prerequisites

Configure test environment

Configure Capybara with a headless driver

References

Password management on the command line

Installation

Getting started

Initialize pass:

Using pass

Adding passwords

Viewing passwords

Removing passwords

Really using pass

Use pass to store secure API keys

Conclusion

References

Rails FormBuilders with TailwindCSS

FormBuilders

Install the Tailwind forms plugin

Create a form builder

Use the FormBuilder

Making TailwindFormBuilder the default

Conclusion

Refernces

Bridgetown and Environment Variables

Background

dotenv

Bridgetown is not Rails

Setting Env Vars

Keeping API keys out of git

Reading Env Vars

Why did you do this?

Re: Learning to code

Immutable ActiveRecord Models

Why?

How?

Creating and updating users with omniauth

Switching to Zeitwerk

Zeitwerk loads constants based on paths

Make sure all your acronyms are properly inflected

Finding problem classes

Initialize `pass`:

Really using `pass`

Use `pass` to store secure API keys