DEV Community: Thomas Hareau

How to release a new gem version in thirty seconds

Thomas Hareau — Tue, 25 Jul 2023 13:01:52 +0000

A while back I wrote a post explaining how we release new
versions of safe-pg-migrations.
The process is manual and a bit annoying to do.

I recently experimented process automation through GitHub action. Though there is one drawback in term of security,
new gem versions can be released in two clicks.

The Anatomy of a Release

A release typically comprises three essential elements: the version number, the .gem file and a "release" on the project's repository.

The Version Number:

Every release is tied to a version number. This numbering scheme, adhering to the Semantic versioning policies, follows the format of MAJOR.MINOR.PATCH, where:

MAJOR version is incremented for backward-incompatible changes.
MINOR version is incremented for backward-compatible additions.
PATCH version is incremented for backward-compatible bug fixes.

By adhering to Semantic Versioning, developers and users can quickly grasp the extent of changes introduced in a release and make informed decisions about compatibility and adoption.

The .gem File:

The .gem file is a crucial artifact generated from the gem's source code using the "gem build" command.

This file encapsulates the gem's functionality. Publishing the .gem file on RubyGem allows developers to access, install, and utilize the gem effortlessly within their own projects. It serves as a packaged version of the gem, ensuring that others can benefit from the latest updates and improvements made to the codebase.

The GitHub Release:

In conjunction with the .gem file, a "release" on GitHub plays a vital role in documenting and showcasing the changes
introduced in the new version.

This release typically contains detailed release notes that highlight the key enhancements, bug fixes, and any breaking changes. By organizing pull requests and commits into meaningful sections, the release notes provide valuable insights into the evolution of the project. The GitHub release is also linked to a specific git tag, making it easy for developers to identify and access the precise version of the codebase associated with the release.

Prerequisite: a bit of tooling

A few configuration are needed to automate releases correctly.

Release notes automations.

We first need to have a way to automate release notes. For this, we can use the configuration file .github/release.yml. This file will tell to GitHub how to classify pull-requests into sections and generate release notes from them.

In my example, I'm using the following release.yml

changelog:
  exclude:
    labels:
      - ignore-for-release
      - dependencies
  categories:
    - title: Breaking Changes 🛠
      labels:
        - breaking-change
    - title: "Fixes :bug:"
      labels:
        - bug
    - title: New Features 🎉
      labels:
        - "*"

Depending on the label set on pull requests, GitHub will group messages together in different sections.

For example, when using the previous configuration the following release notes would result.

Version bump automation

Each new release requires the new version to be written manually. For this, I've created a rake task which automatically bumps the version.

# frozen_string_literal: true

require 'rake'
require 'version'

task :version do
  puts MyGem::VERSION
end

task :bump, [:type] do |_t, args|
  type = args[:type]
  raise 'Must specify type: major, minor, patch' unless %w[major minor patch].include?(type)

  puts "Old version was #{MyGem::VERSION}"

  major, minor, patch = MyGem::VERSION.split('.').map(&:to_i)

  case type
  when 'major'
    major += 1
    minor = 0
    patch = 0
  when 'minor'
    minor += 1
    patch = 0
  when 'patch'
    patch += 1
  else
    raise 'Unknown version'
  end

  new_version = [major, minor, patch].join('.')

  new_content = File.open('lib/version.rb', 'r') do |f|
    f.read.sub(/VERSION = '#{MyGem::VERSION}'/, "VERSION = '#{new_version}'")
  end

  File.open('lib/version.rb', 'w') do |f|
    f.write(new_content)
  end

  puts 'Bumped to ' + [major, minor, patch].join('.')
end

This script can be called as follow rake "bump[${release_type}]" (where $release_type can be either patch, minor or major, following semantic versioning policies).
The new version number will be computed and replaced in the code automatically.

NOTE

A few tweaks are probably needed if you intend to use it in your code, like the name of the gem and the file location.

Releaser script

We can now create the releaser script. It has to be stored in the .github/workflows/ repository, under any name that suits your needs.

The file looks like this:

name: Create a new release

on:
  workflow_dispatch:
    inputs:
      type:
        type: choice
        description: Type of release
        options:
          - patch
          - minor
          - major

jobs:
  release:
    if: ${{ github.ref == 'refs/heads/main' }}
    permissions: write-all
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3
      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: 3.1.2
          bundler-cache: true
      - name: Configure git, bundle and gem
        env:
          GEM_HOST_API_KEY: "${{secrets.GEM_HOST_API_KEY}}"
        run: |
          git config --global user.email "youremail@email.test"
          git config --global user.name "YOUR NAME"
          bundle config unset deployment
          mkdir -p $HOME/.gem
          touch $HOME/.gem/credentials
          chmod 0600 $HOME/.gem/credentials
          printf -- "---\n:rubygems_api_key: ${GEM_HOST_API_KEY}\n" > $HOME/.gem/credentials
      - name: Bumping
        run: |
          bundle exec rake "bump[${{ inputs.type }}]"
          bundle install
      - name: Commit and push
        run: |
          git commit -a -m "Bump version to v`bundle exec rake version`"
          git push
      - name: Build gem
        run: gem build *.gemspec
      - name: Push gem
        env:
        run: gem push *.gem
      - name: Create release
        env:
          GH_TOKEN: ${{ github.token }}
        run: gh release create "v`bundle exec rake version`" *.gem --generate-notes --latest

Let's dive a bit inside:

on: workflow_dispatch

The action is a manual action. Users will need to trigger it manually, through the workflow tab on GitHub.

on:
  workflow_dispatch:
    inputs:
      type:
        type: choice
        description: Type of release
        options:
          - patch
          - minor
          - major

The inputs part indicates that the workflow will have an input, named "Type of release" and allowing three different options. It will be passed directly to the rake task to bump the version of the gem.

Steps

The steps part defines a list of actions to be run sequentially.

Most steps are straightforward and easy to understand. In particular:

Configure git, bundle and gem

This step configures git and ruby so that we can push a new commit with the version, and push the gem to RubyGem. In particular, the four following lines are storing gems credentials :

mkdir -p $HOME/.gem
touch $HOME/.gem/credentials
chmod 0600 $HOME/.gem/credentials
printf -- "---\n:rubygems_api_key: ${GEM_HOST_API_KEY}\n" > $HOME/.gem/credentials

This requires that a RubyGem API key is stored in GitHub repository secrets under the name GEM_HOST_API_KEY.

Bumping executes the rake task we created before, and executes bundle install to update the version in Gemfile.lock file.
Create release will generate a new GitHub release. The option --generate-notes --latest are given, to mark the release as the newest version, and to generate release notes using the configuration we defined before. Also, as the git tag corresponding to the version does not exist yet, GitHub will create it automatically.

The 2FA issue

RubyGem strongly recommends to have Two Factor Authentication (2FA) activated. In term of security this is an excellent idea. However, it is an issue when scripting because gem push will require a One Time Password (OTP) code.

The risk, and why it can be acceptable

OTP codes are used to strengthen password authentication. OTP protects from password thefts: if a
password has been leaked, an OTP code is still required and prevents authentication.

The strength relies on the difficulty to leak an OTP code. It is indeed regenerated every 30 seconds, and invalidated
after the first use. And even if the user is victim of a phishing attack, the OTP code will not be usable for another
authentication.

That said, if OTP codes cannot be leaked, it is not the case of the OTP secret. Codes are generated from a unique secret
that should stay hidden. If an attacker has access to the secret, they can generate as many valid code as they want. Therefore the secret has to be stored securely.

The tweak I am proposition stores the OTP secret alongside the API key, in the GitHub secrets vaults.
Mainly, this bypasses 2FA: RubyGem will think that the authentication is strong whereas it
actually has the same level of security as a standard authentication.

There is also the possibility of having an attack on GitHub secrets vaults. If it would leak, both the API key and
the 2FA secret would be stolen, making authentication possible on RubyGem.

On my side, I believe that the implied security risk is acceptable: GitHub is definitively trustworthy in term of security, especially on such a sensitive topic as the secrets vault. In my opinion, the likeliness of a a data-breach of the vault is extremely low.

However, personal GitHub account should be well secured (using Two Factor Authentication and a strong and unique password). Anyone gaining access to a maintainer's GitHub account would be able to release new versions of the gem.

The 2FA tweak

To have 2FA working, we will modify the push step to the following:

steps:
      - name: Add OTP generator
        run: |
          gem install rotp
      - name: Push gem
        env:
          GEM_OTP_SECRET: "${{secrets.GEM_OTP_SECRET}}"
        run: gem push *.gem --otp `rotp -s "${GEM_OTP_SECRET}"`

This will generates the OTP code depending on the TOTP secret, that should be stored in GitHub secrets as well.

Conclusion

We can now release a in 2 clicks.
The workflow will take less than a minute to run, and several users could release under the same global account.

If you want to try it out, feel free to fork the test repository.

Cover picture by Peps'

Accessing state of an uncontrolled form

Thomas Hareau — Sun, 29 Jan 2023 17:37:10 +0000

Controlled forms are convenient because they give a thin control over the different values. However, they tend to be a bit verbose.

On the other side, uncontrolled forms are often shorter and more readable. They let the browser natively handle the component, reducing the needed boilerplate for controlled components.

Note: a French translation of this article exists on my website

Controlled form

For example, let's take this form. We have four inputs, "diet" being disabled if the "presence" checkbox is not ticked.

Here we need to handle all fields manually. This works, but seems to be unnecessary when HTML alone handles this perfectly well.

Uncontrolled form

Let's take the same example, using an uncontrolled form.

const action = (event) => {
  event.preventDefault();

  const data = new FormData(event.currentTarget);
  console.log("Saving...", Object.fromEntries(data.entries()));
};

const App = () => {
  return (
    <div className="App">
      <form onSubmit={action}>
        <fieldset>
          <legend>Identity</legend>
          <div>
            <label for="name">Name : </label>
            <input id="name" name="name" defaultValue="Chris" />
          </div>
          <div>
            <label for="email">Email : </label>
            <input id="email" type="email" name="email" defaultValue="chris@email.test" />
          </div>
        </fieldset>

        <fieldset>
          <legend>Event</legend>
          <div>
            <label for="presence">Presence : </label>
            <input id="presence" type="checkbox" name="presence" defaultChecked />
          </div>
          <div>
            <label for="diet">Diet : </label>
            <input id="diet" name="diet" />
          </div>
        </fieldset>

        <button type="submit">Submit</button>
      </form>
    </div>
  );
};

We can notice two main changes here:

we have way logic (and thus fewer bugs). We don't need form values before the submit action, so why storing them?
action reads value directly from the event. As it does not use React states, we can extract it outside the component, and split our code more easily.

Oh, and maybe a last point: we don't handle the disabled field anymore. Let's do it!

Accessing state using uncontrolled forms

We need to read the current value of the "presence" checkbox to disable the "diet" field. Using controlled components, it is easy. Here, less so.

A solution to this problem is to add an onChange listener on the form itself. It will provide read-only access to the current values of the form.

const App = () => {
  const [formData, setFormData] = useState(new FormData())

  return (
    <div className="App">
      <form onSubmit={action} onChange={(event) => setFormData(new FormData(event.currentTarget))}>

And then, on the diet field, we add:

<div>
  <label for="diet">Diet : </label> 
  <input
    id="diet"
    name="diet"
    disabled={formData.get("presence") !== "on"}
  />
</div>

Which gives this final result:

Database... or Goose?

Thomas Hareau — Mon, 28 Jun 2021 13:56:00 +0000

Once a platform reaches a certain amount of usage, database migration can quickly bring a platform down. For example, adding a column, if not done carefully, can block entirely lock a table for a few minutes.

At Doctolib, we developed safe-pg-migrations, a tool which automatically makes migrations safe. In 2018, we open-sourced this library. From an internal solution developed by our team only, it is now used on other projects, and even received contribution from external authors.

To acknowledge, we decided to give safe-pg-migrations a logo. Our design team came up with two propositions.

We would like you to help us choose the best logo. To do so, answer to the survey on the tweet bellow:

// Detect dark theme var iframe = document.getElementById('tweet-1409421282773549058-68'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1409421282773549058&theme=dark" }

How to release a new gem version

Thomas Hareau — Thu, 18 Mar 2021 15:13:45 +0000

At Doctolib, we proudly maintain and develop the gem safe-pg-migrations, which makes migrations safer. Once in a while, after each relevant update, we release a new version.

The process is easy, but not well documented. I usually rely on my bash history to remember the different steps, but today, history | grep "gem push" gave me an empty result. So I decided that a blog post was more reliable than a self-deleting history.

The process presented below is the same for creating or updating your gem. In the example below, we use safe-pg-migrations and 1.2.0 as examples, they will need to be adapted to fit your needs.

Be sure of what you are going to publish.

Before starting the release process, you need to make sure the code you are about to publish is working as expected.

Have a look at the git history, make sure you are updated with the latest version of the main branch. Launching the tests is also a good idea.

Bump the version of the gem

First: update the version. It is specified in the gemspec file. You will need to open it with an editor:

vim safe-pg-migrations.gemspec

And then update the version definition:

  s.version = '1.2.0'

When choosing the new version, remember to follow the semantic versioning policies.

The version is also duplicated into Gemfile.lock. You can only update the CHANGELOG, if you maintain one.

Once the version is bumped, you can commit the changes:

git commit -m "v1.2.0"
git push

Tag the new release

Once the version is updated, even if non-necessary, it is good to create a new tag. To do so, in your terminal, execute:

git tag -a v1.2.0 -m "Changelog..."
git push origin v1.2.0

Tags are useful because they help other developers to see exactly which version of the code is associated to which version.
Every release can be fetched with git tag.
A specific version can be checked-out with git checkout v1.2.0.
Tags will also be listed in a specific page on GitHub, highlighting each release.

Publish the new release to RubyGem

Now the version was bumped and tagged, you can now proceed by publishing it to RubyGem.

First, you need to build a .gem file:

gem build safe-pg-migrations

Once created, the file can be finally published. If you haven't done it already, you will need to create an account at https://rubygems.org/users/new.

gem push safe-pg-migrations-1.2.0.gem

This last step will take a few seconds to complete. You should receive an email shortly after, confirming that a new release was published.

Congratulation, you have published a new version of your gem!

If you find any most effective way to release, make sure to let us know in the comments!

Cover picture by Peps'