Sensitive data encryption in Rails

Recently I received a request to encrypt some part of data that was already in my client’s app. I don’t want to share too many details, so let’s say it was few columns of an existing table that already contained some data.

So my task was to enrypt all existing records, delete non-encrypted data and save all future records as encrypted. Sounds cool!

Lockbox
What I needed was a simple ruby gem that would help me with all above tasks. I found Lockbox or - the Lockbox found me! I’ve read about it some time ago in RubyWeekly newsletter. It offers migrating existing data, encrypting columns, decrypting fields on the fly if you need to display them somewhere in the views, etc.

The Plan
Since our app is used 24/7 we didn’t want to put it in the maintenance mode, we had to split update into few steps without causing any downtime.
I came up with this solution:

1. Add Lockbox to gemfile, add new columns to persist encrypted data, update model to migrate existing data
2. Add ignored_columns attribute to model (to remove columns in next step without any downtime)
3. Remove non-encrypted columns from table
4. Remove ignored_columns attribute

This required 4 pull requests creation, 1 task execution and zero downtime :)

Execution
I started by adding gem 'lockbox', '~> 0.4.6' to the Gemfile and generating Lockbox master key.

Lockbox.generate_key

This key had to be added to all secrets (in dev, staging, production and test environments). I also created lockbox.rb file inside initializers/ folder.

After that Lockbox was ready to go, so I could start work on encryption of my desired model. Let’s say that I wanted encrypt our clients’ correspondence data. Inside Client::Address model, I had to encrypt following fields: street, city, postal_code, country, phone_number

So inside models/client/address.rb I had to add:

class Client::Address < ActiveRecord
  encrypts :street, :city, :postal_code, :country, :phone_number, migrating: true
end

Notice last part of it - migrating: true. This is crucial for next step - migrate existing data. But before that I had to create new columns for encrypted data. So I came up with following migration:

def up
  add_column :client_address, :street_ciphertext, :text
  add_column :client_address, :city_ciphertext, :text
  add_column :client_address, :postal_code, :text
  add_column :client_address, :country, :text
  add_column :client_address, :phone_number, :text

  change_column :client_address, :street, :string, null: true
  change_column :client_address, :city, :string, null: true
  change_column :client_address, :postal_code, :string, null: true
  change_column :client_address, :country, :string, null: true
  change_column :client_address, :phone_number, :string, null: true
end

At this point no null: false constraint could take place - adding not_null columns to existing records would paralyze app. Also null: true had to be added to existing columns - from now on we would save new records only to _ciphertext columns.

After successful migration, there was only one thing missing. Data migration itself!
Thanks to Lockbox, this is done by running simple command in rails console. After merging above changes, I had to ssh to production environment and run

Lockbox.migrate

At this point all existing data was migrated to _ciphertext columns. But I needed more - I needed to remove non-encrypted columns with sensitive data saved in plain text!

Step II
First of all I removed migrating: true from models/client/address.rb model - this was no longer needed.
Next thing was migration:

def up
  change_column :client_address, :street, :text, null: false
  ...
end

For all _ciphertext columns I added not_null constraint, since we’re already saving records to that columns and none of them was empty anymore.
And the most important part of this pull request was adding ignored_columns to Address model. Since in next PR I wanted to remove all unnecessary columns this was crucial at to add this at this point.

class Client::Address
  ...
  ignored_columns: %w[street city postal_code country phone_number]
  ...
end

Step III
This part is only about removing non-encrypted columns. So it included only this migration:

def up
  remove_column :client_address, :street
  remove_column :client_address, :city
  remove_column :client_address, :postal_code
  remove_column :client_address, :country
  remove_column :client_address, :phone_number
end

Thanks to ignored_columns this could be deployed without causing any downtime to app. From this point there were no decrypted clients’ address data in our database.

Last step was to remove ignored_columns attribute from the Address model.

Conclusion
Encryption was definitely worth adding and considering fact that this update was deployed at Friday after 2a.m. with no issues, this shows that process is very easy.
At least on that set of data that I had to encrypt.

Comments

[Kudakwashe Paradzayi]

Do you have an idea why Client::Address.create(args) works but Client.addresses.create(args) gives an error

NoMethodError (super: no superclass method `seed=' for #<Account:0x0000aaab0b8fbf90>).

I just used Client and Client::Address to remain in the context of your article, but this is happening on models that have a one to many relationship. Help.

[rbglod]

Without seeing code of your models I can't really help, sorry. Might be everything. Make sure that Client is connected with Address in a correct way, provide 'class_name:' attribute after has_many :addresses. Does seed method/attribute tells you something? Are you using it somewhere?

[Ian bradbury]

Great write up. Thanks. I’ve book marked - I know I’ll need this at some point.

[rbglod]

Thanks for spotting that! Fixed :)

[Druid]

Amazing read!

[Pere Joan Martorell]

Thanks for the article, but this requires more or less 4 deploys, 1 for every step. Is it possible to do all the migration in a single deploy?

[rbglod]

Nope, that's the clue - every change should be deployed separately. That gives us ability to keep app up and running during all deployment process.