A couple years ago, I broke a dashboard at work.
As part of building a new feature, I changed our database schema. I carefully laid out a plan to add some new columns and migrate existing data. I made sure the changes to the database would work with both new and old application code and the feature could be deployed without any downtime.
I forgot to consider that our application wasn’t the only consumer of our data. Our data team maintained dashboards that the company used to inform product decisions and make financial projections. Those dashboards expected our database schema to look a certain way, and when I made the changes without talking to the data team, one of those dashboards broke.
After we fixed the dashboard, I thought about how I could prevent myself from making the same mistake in the future. I decided needed a reminder each time I made a change to our database schema. To get that reminder, I reached for the tool I was most comfortable with and spun up a new Rails app.
There’s a lot to a Rails app
My goal with the app, DiffAlert, was to send an alert to Slack each time someone made a change to db/structure.sql on the master branch of our company’s application. I saw that GitHub had webhooks and that the push event sent along data about commits, including which files were changed.
In a few hours, I spun up the app, created an endpoint for the GitHub webhook events, and wrote the code to tell whether or not db/structure.sql changed in a given push. DiffAlert’s core logic was complete. Then the real work started.
Next, I had to figure out how to send an alert to Slack. We were already using Slack’s Email app, so I signed up for a new Mailgun account and created an email template with Action Mailer. Then I had to deploy DiffAlert somewhere, so I created a new Heroku application and did some tweaking to get everything properly configured with my workflow.
Besides db/structure.sql, we’d also want to monitor a few other files in our codebase, so I started building a UI to configure alert settings. I’d need authentication, so I designed the data model and created login and sign up forms. Then I needed another view to show all the alerts, and I needed forms to create new alerts and edit existing ones. I needed background jobs so that longer processes like webhook parsing and email sending wouldn’t hold up regular web requests.
DiffAlert chugged along for about a year and a half, reminding my team in Slack when we made changes to our database schema. Each time, we reached out to the data team and didn’t break any more dashboards.
DiffAlert was a side project, so when I left the company, they needed to decide if they would continue sending a former employee their GitHub metadata, maintain their own instance of DiffAlert, or stop receiving alerts. They understandably decided to stop receiving alerts.
GitHub Actions helped me focus on the problem
I work for GitHub, and when I learned about GitHub Actions, I wondered if I could replace DiffAlert with a single Action. A GitHub Action is code, written in any language, that runs in a Docker container when a specified event happens in a repository. I saw that push events could trigger GitHub Action workflows. I also saw that there was a GitHub Action that could post messages to Slack. I started writing Modified File Filter.
First, I needed a Dockerfile. I wanted to write my Action in Ruby, so I used the FROM instruction to create a Docker container from an official Ruby base image. I wrote some LABEL instructions so that my Action would show up correctly in the GitHub Actions visual workflow editor. Finally, I specified which folders the Docker container would need access to and pointed the ENTRYPOINT at an executable Ruby script.
# Dockerfile
FROM ruby:2.6.0
LABEL "com.github.actions.name"="Modified File Filter"
LABEL "com.github.actions.description"="Stops a workflow unless a specified file has been modified."
LABEL "com.github.actions.icon"="filter"
LABEL "com.github.actions.color"="orange"
ADD bin /bin
ADD lib /lib
ENTRYPOINT ["entrypoint"]
The executable Ruby script delegates most of the work to a plain old Ruby class, PushEvent, which parses the event data from GitHub and answers whether a file at a specific path is modified. When a push event modifies the file, the Action exits with a 0 status to trigger the next Action in a workflow. When a push event doesn’t modify the file, the Action exits 1 to halt the workflow.
# bin/entrypoint
require_relative "../lib/push_event"
file_path = ARGV.first
push_event = PushEvent.new(File.read(ENV.fetch("GITHUB_EVENT_PATH")))
if push_event.modified?(file_path)
puts "#{file_path} was modified"
exit(0)
else
puts "#{file_path} was not modified"
exit(1)
end
I didn’t need to configure emails or figure out how to integrate with Slack since I could lean on the existing GitHub Action for Slack. I didn’t need to design any UI or authentication because that was all handled by GitHub. I didn’t need to configure databases or background jobs. I didn’t need to spin up a Heroku application and set up deployments.
Starting smaller
Rails has a reputation for being a great framework for validating ideas. You can use Rails to build a blog in 15 minutes! Because I love working with Rails, I often reach for it first when I have an idea.
But even with all the magic that Rails provides, most apps need a whole bunch of things — like authentication, UI, background jobs, email sending, deployment — that aren’t unique to my idea. Next time I have an idea, I’ll look for ways to write less code and maintain less infrastructure, at least to get started.
Cover image by Iain Farrell on Flickr.
Top comments (10)
Love this. We live in exciting times, the internet is rich with pluggable tools resembling larger LEGO pieces.
Appreciate the reminder to focus on the problem, remaining open minded on tooling.
Well said! I totally agree. Thanks, Andrei!
The first problem looks a lot to me like the following => Your DB = Your API.
That's the very big problem with rails. Everything in rails lead you to think that data in DB, data in API responses (through render json: @model) and data in API requests (through: Model.new(params[:model])) is the same.
Well it's not.
That's why, even if many voices in rails community are against
1) Template views (like json.jbuilder) with aliasing every property manually should always be favored over render json: @model or other magical things (if you have rest API, or you can use graphql, it's even better)
2) Requests should be handled by form objects then form objects would translate requests data into model data, acting as a proxy, leaving the API intact if you rename some columns (or you can use graphql)
My point is: you shouldn't have to monitor db/structure.sql because changing the SQL structure should not have any impact on API contract.
Ok! I disagree and still like Rails. Interesting take, though.
Well, I agree with the final lesson. However, sometimes there's no solution than to do more job than the expected.
Not sure about the dates but could it be GitHub actions weren't available by the time you coded DiffAlert?
Anyway, both things are nice and I can be sure you learnt something real good while doing DiffAlert, didn't you?
There's this book called "Building Software Products in a Weekend" and the guy basically says that you should first look for something built(do an intensive search), if it exists buy it(if costs money), then if doesn't exist, build it. But again, surely GH Actions were not a thing by then.
Thanks, Francisco!
That's true: GitHub Actions weren't around when I worked on DiffAlert. I bet there was another solution out there at the time that would have been less code and infrastructure than a Rails app though. I wonder if I could have pointed GitHub webhooks at Zapier or another similar service.
Agreed! Yes, I had a blast building DiffAlert, and I learned a whole bunch in the process.
Interesting post, and absolutely agree with the overall message "write less code and maintain less infrastructure".
Going back to the original problem - the dashboard breaking because of the schema change - did you consider using database views (at least for the dashboard)?
That way you could have made the schema change (including updated views) and the dashboard contract would have been maintained.
The scenic gem even lets you do views "the rails way".
Thanks, Ewan! I hadn't considered database views, but that's a great point -- I'll definitely think of them the next time I encounter a similar problem.
Love this and WISE conclusion.
Thanks for sharing
Thanks for the article! We will look into doing something just like you did.