I remember reading many times over this piece of advice given to new, aspiring developers : « Go ahead! Read some source code, then contribute to open source projects! ». Truth be told, hearing this right after attending a 2-month coding bootcamp kind of felt like being pushed to try and climb a mountain with slippers and a swimsuit.
In spite of this, barely three years forward and a few apps later, a good friend of mine and I just released our very first Ruby gem, abyme. From the moment we rolled up our sleeves, up to our first release on rubygems, we just learnt so much along the way we felt we ought to share this experience, in the hope of empowering others to do the same.
Why you should build a gem too 👊
Before we dive in, let's quickly review a few reasons as to why we think building a gem is good for everyone - including you.
An obvious first reason would simply be to give back to the community. After all, most tools we use are open source : not only are they free, but for the most part, they're being actively maintained by contributors like you and me on their free time. As much as we could, sharing the fruit of our labor to help others feels pretty legit.
Another pretty obvious reason is the amount of knowledge you're likely to acquire by doing it. It's one thing to build applications using tools built by others for us ; it's a whole other thing to build the actual tools we use everyday ourselves. Among other things, gems involve quite a bit of metaprogramming, a rigorous file structure and file loading patterns ; you'll often have to resort to all kinds of tricks to make things work, and you will probably have to dig deep into ruby's closures and advanced OOP patterns. Finally, if you code a gem for Rails, you will no doubt have to read through lots of source code to get a grasp of how things actually work behind the scenes. All this becomes priceless knowledge when it comes to using and debugging these tools in your day to day job.
Finally, let's not forget another aspect of building stuff : whether it will get used or not, you'll probably get a nice ego boost out of your achievement. Making something work will make you proud and more confident in your programming duties. And if it does get used, a little fame doesn't hurt either.
Now that you know why you're here, let's go over the journey that awaits.
⚠️ Disclaimer : the rest of this article is not intended as a step by step tutorial on how to make a gem, but rather as a kind of retrospective on the different steps involved in building a gem, their associated challenges, and how we approached them as first-timers. If you're fine with that, read on !
Entrepreneur 101 💼
Like most journeys, you'll have to have some goal in mind. Think of your future gem like you would any other product : start with a pain. Something that you wish existed, or something that you find yourself coding again and again and that you feel could be DRYed up. Even better : try to identify in your actual work some cool behaviour that could be extracted and still hold its value.
Our first motivation to try and build something came from the frustration that often arise when resorting to nested attributes in Rails, and the absence of built-in Javascript behaviour to dynamically add fields on the fly (more details on the subject here). Solutions do exist, but they're either impractical, dated, or require quite a bit of configuration. Therefore our goal was simple : offer a plug-n-play approach to dealing with nested attributes in Rails.
First implementation 📝
Once you know what you're building, just build it. Scaffold some demo app, and implement the behaviour you need as you would in a normal application. Once things work as they should, and if you haven't done TDD (I won't judge), you should write tests, so as not to break behaviour in your coming experiments.
In our case, nested attributes were already implemented in Rails through different modules (namely ActiveRecord and ActionView) so the basics were easy to setup. The JavaScript part though is not part of Rails's core, but many tutorials exist. One of those is this great Drifting Ruby episode, which we took as a base to build our demo application. In just a couple of hours of pair programming, our application was behaving the way we wanted.
While we successfully achieved our main goal front-end wise, the way it worked at that point in the back-end was still a bit tedious : for each new nested association, configuration needed to be copy-pasted in every model, every controller, every form. Plus, there was no way to easily control where the fields would appear, nor to make a difference between fields for already existing associated items or fresh ones created through our JS behaviour. Time to enhance our development experience.
DevX is your new UX 🦾
If we think of a gem as we would a product on the client perspective, our goal is to make sure it's intuitive, efficient and flexible. Think of it like the UX/UI aspect of a website ; only this time your clients are developers like you.
This is probably the most fun part : it all revolves around how we want to interact with our code : what should we call this method ? Should we pass arguments or a block to it ? Should it be an instance method of something ? To what extent should we allow customization ? Where will we put the configuration ? It also means anticipating a few different use cases, potential generators, while as much as possible, respect conventions, and give it a feel as close as possible to native Ruby/Rails code.
An interesting part of this process for us was the naming of our core method : the one that generates the form fields for the nested association. Just so you can have a feel of how it evolved, here's what the first iteration looked like :
<%= form_for @project do |f|
<%= abymize(:tasks, f) %>
<% end %>
After a few months of use in real conditions, we decided it was really not intuitive, and it lacked the native Rails feel too much to our taste. We decided we wanted something like this instead :
<%= form_for @project do |f|
<%= f.abyme_for :tasks %>
<% end %>
This had a much nicer feel. Problem was, it was much, much more difficult to implement, since it no longer meant having a simple view helper method. That was one of the toughest obstacle we had to overcome : it meant digging into what f actually stands for, find where it was originally defined, and how to add behaviour to it. The only way out was to - you probably guessed it - dig into some source code.
We eventually directed our search towards the famous simple_form gem source code, since we knew that it was fiddling with the same mysterious f object - which turned out to be an instance of ActionView::Helpers::FormBuilder class. It gave us a few hints (most notably in the way we should register our method) but it wasn't enough to make it work ; it was only after reading through Rails class definition (particularly line 2241 and line 2629) that we found the solution to our problem : we just had to call our existing helper method on the instance variable called @template. Happy devs we were.
Be mindful though : just like with any product, there's an infinite way of providing satisfaction, so you may want to just stick with some basic functions at first (like an MVP !). Open source will also mean that anyone will be able to contribute and add some other behaviour you would not have thought of, so it can actually be a good thing to voluntarily leave some blank space.
Extraction ⛏
After coding all the features we wanted came the million dollar question : how the hell were we going to turn this into a gem ?
You probably know that real rubies need to be extracted from actual rock, which is a tedious and dangerous process for the stone. While not particularly dangerous (as long as you have tests), the software version of the gem extraction remains a pretty delicate exercise. A good starting point is this excellent Drifting Ruby episode. Dave Kimura takes us through the first steps of generating the gem repository and moving the code piece by piece from inside the application to the lib folder, then to where it belongs in the gem’s own repository.
It only goes so far though ; chances are your gem will need to extend some native behaviour, and it will be your job to find how to do that. Luckily, thousands of gems exist in the wild, so your best bet will be to, again, get inspiration from what others did before you. My main education consisted in music harmony, and during my jazz years, I was taught that there was no better education than learning from the best : until today, I've spent countless hours transcribing solos from my favorite players by hand (I even wrote a small application on my free time during lockdown to allow people to freely share their own transcriptions).
In our case, we just went and looked for gems that looked remotely similar to ours, and studied their code. Since our gem required code to run in models, controllers, and views, we actually had to turn it into a Rails engine, which offers several benefits over a simple gem (most notably when it comes to testing, since an engine allows you to have a dummy app in your test directory for - take a guess - test duties). Then loading the different pieces was as easy as :
module Abyme
class Engine < ::Rails::Engine
isolate_namespace Abyme
config.after_initialize do
ActiveSupport.on_load :action_view do
include Abyme::ViewHelpers
end
ActiveSupport.on_load :action_controller do
include Abyme::Controller
end
end
end
end
You probably wonder where we include the Abyme::Model part ; this may very well change in the future but we decided not to load it by default, so that users include it manually in their model (unless using our generators) like so :
class Task < ApplicationRecord
include Abyme::Model
has_many :comments, inverse_of: :task, dependent: :destroy
abymize :comments
end
Finally, we shamelessly copied SimpleForm 's way of registering their simple_fields_for method on Rails's original FormBuilder class :
module Abyme
module ActionViewExtensions
module Builder
def abyme_for(association, options = {}, &block)
# Call to our #abyme_for view helper, defined in Abyme::ViewHelpers
@template.abyme_for(association, self, options, &block)
end
end
end
end
# The registration takes place below
module ActionView::Helpers
class FormBuilder
include Abyme::ActionViewExtensions::Builder
end
end
Packaging 📦
Your efforts are about to be rewarded. Everything works, your tests are all green, and you're proud as a peacock. All you need to do now is just ship the damn thing.
Fortunately, this comes down to a very simple command (documented in Bundler's documentation) : rake release . You'll just have to update your gem version appropriately before running it, and it will take care of releasing the gem for you (you'll need to register an account on rubygems though). In a few seconds, you'll be ready to just bundle add your gem !
We need to talk about JS
I have to admit we kind of struggled with the JavaScript part of the gem, which is really poorly documented in the context of Rails gem building. Being responsible for the packaging duties, I actually broke the gem several times trying to make it work.
After doing some research it turned out there is no real way of including Javascript code through a gem. The JS code needs to be packaged into a npm or yarn package of its own. However, nothing holds you from using your gem repository as your yarn package repository as well, which is the approach we took. Naming the package like the gem, it allows installation with bundle add abyme and yarn add abyme which makes for a nice symmetry 👩🏾🤝👩🏼
On the packaging side, I learnt the hard way that you can't just leave some JavaScript in a file and expect it to work right off the bat. I had to try different packaging strategies before making it work (mainly because of Babel-related stuff that is outside the scope of this post). Just remember you'll need to use a packaging tool such as Microbundle to allow your ES6 (or newer) syntax to be compiled correctly by Webpack.
Hello World ! 🌎
At this moment, you may feel like you finally reached the top of the mountain. Unfortunately there's still quite a lot of stuff to be done before introducing your baby to the community. After taking care of packaging your gem and shipping it to rubygems, your to-do list should contain the following things :
- Have and maintain maximum test coverage
- Write some good and thorough documentation
- Build a demo app to showcase your gem and its usage
- Write a step by step tutorial (optional, but quite valuable)
- Setup continuous integration tools
- Grab a few badges to inspire confidence (code coverage, maintainability, tests passing...)
- Generate a
CHANGELOGto keep track of all changes and code additions
I can assure you from our recent experience that these things take time. Probably even more time than writing the gem itself. You may have written the most revolutionary tool, but if documentation is lacking or if the README sucks, chances are no one will bother to even try using it. Which kind of defeats the purpose of a community-driven endeavour.
Wrapping up 🎁
Time to get a little personal. Since writing abyme, I can really feel how it changed my way of approaching new features or refactoring in my applications. Without giving much thought to it, I now tend to better identify recurring and reusable pieces of logic, then implement them in a way that will make future extraction as straightforward as possible - when appropriate of course. I also have a better feel for when little bits of metaprogramming can sometimes really make things much more tidy and flexible. I think I have a much better understanding of how certains parts of Rails work, and know how to safely override or extend some gems behaviour when I need to. Overall I just feel much more confident as a developer.
Technical skills aside, it also revealed how hungry I was for such challenges. I love building tools, sharing knowledge, and open source represents such an infinite pool of both resources. I don't think I could envision my life without a reasonable time dedicated to this kind of contributions anymore. Coming from a creative industry, I naturally feel at home when giving life to fresh ideas that I think are worth sharing. And I happen to have a few ones coming 😎
Last but not least, I think the most rewarding thing is to observe that other junior/mid developers around me have started to adopt a mindset of I can do it too. And that's what I think open source should be about : share with others the will to just try and contribute.
Resources 📚
I'll keep this list updated with worthy additions.
- Bundler doc
- Rubygems guide
- Rails engines documentation
- Drifting Ruby (the gem series)
- Eloquent Ruby (particularly for the chapters about DSL and metaprogramming)
- Ruby closures
About us 🤓
Romain and I are two alumni from Le Wagon, a famous coding bootcamp which you might have heard of. We both attended batch #177 during summer 2018 in Paris, and are now teaching there while working freelance as fullstack web developers.
Cover picture by Jason D
Top comments (14)
Great article, but how do you differentiate your gem from the already pretty famous "Cocoon"?
In all honesty, we didn't feel like doing so. Neither of us ever liked working with
cocoonbut still did because it was the only available tool to help with the issue at hand. I think contributing to an existing codebase makes sense for projects you love and wish to support and enhance, but here we really wanted to start from scratch with a completely different approach.That being said, it's one of our main references for this project, and it helped us tremendously in writing some parts of the gem. Like I said, learn from the best ; (the missing part was "Then, try and make it better !" 🤓)
Hey Roman ! Thanks ! Our take on the issue is quite different from cocoon. We're providing users with a complete wrapper around
nested_attributes, which includes :cocoonis implemented injquery, whereas we chose to implement everything withstimulus. Have a look at the doc for more details 😬Alright cool! I definitely try it out next time in need for nested forms! ✌️
Cool to use stimulus, it's definitely an underrated framework!
Excellent article Louis and Romain!
It's great that you shared the whole process, from the pain to the release while still describing all the code challenges you had to solve ✨
Congrats on this new gem!
Incredible article! Thanks for taking us on your journey and for sharing the way you came about this gem. Love the "let's do it and make it" attitude.
I agree, DevX is so important.
Can't wait to start using it and to see more people adopt the gem. Congrats to both of you!
Very cool. Did you consider a task like rails ships with, which detects yarn and runs? It's built into rails 6 I think, so is an official rails friendly pattern as far as I'm concerned.
Thanks Lewis ! That's good to know ; do you think it would be appropriate to use this pattern for the whole javascript part of the gem ?
Hey Louis,
I was thinking just for initial installation.
Not sure what you meant though. Sorry for slow reply.
Really enjoyed reading.
Excellent ! I wish more guides were as well written as this :)
Congrats to you Romain and Louis! Both for the gem and for the description of your journey in this article!
High quality post, thanks!
Excellent ! Amazing job! Thanks for sharing👍