Hanami Shrine - file handling in Hanami

Continuing with the latest streak of Hanami focused posts I am bringing you another example of a common feature and implementation, translated to Hanami.

I recently showed some email-password authentication with Hanami, before that it was progress bar feature, now we will handle image uploads.

As a sidenote, nice thing about writing about Hanami is that you get to use all those beautiful pictures of blossoming trees. This time with a shrine in the background.

Shrine

We will be using shrine and I want to start this post by saying a few words about it.

It is great.

I could end it here, but just to clarify: it is a file attachment toolkit for Ruby applications. It is very flexible and can be used with any ORM, any storage service, and any (relevant) processing library. It is also very well documented and has a lot of plugins. We will be making use of those.

ActiveStorage and Shrine

I worked with ActiveStorage on Rails a lot and I have been frustrated by it many, many times. It is overcomplicated, and does not follow the usual doctrine of Rails, that aims to make developers happy. ActiveStorage does not make me happy. It makes me frustrated. It makes me spend a lot of time on documentation, cause even though I keep using it, due to its design and DSL I cannot, for the life of me, remember multiple important details about it. I need to constantly remind myself how to do certain, even mundane, things with it, despite doing them repeatedly.

Shrine does make me happy. It is stupidly simple. I rarely look up the documentation after working with it on few projects. The documentation is also written well and is very concise, something that a lot of rails guides pages fail at in my opinion.

Hanami with ROM and Shrine

So if you have read my previous posts about Hanami, you probably noticed I am using ROM-RB. As an ORM, rom provides minimum infrastructure for mapping and persistence by design. It presents data with immutable structs. Those structs are disjointed from the layer that persists new data to the database.

You might think "oh-oh", this means we have to go through a lot of hoops to make Shrine work with ROM. But you would be wrong. In fact, you are wrong. Shrine is very flexible and can be used with any ORM, remember? It wasn't always that easy, but with a shrine plugin comes with the gem, but has to be enabled we can make it work, surprisingly easily.

Setup

Once again, we have to start with configuring the tools used. If you have seen previous posts, you can figure out that we need to add a new file in config/providers

#config/providers/shrine.rb
Hanami.app.register_provider(:shrine) do
  prepare do
    require "shrine"
    require "shrine/storage/file_system"
    require "shrine/storage/s3"
  end

  start do
    s3_options = {
      bucket: target["settings"].s3_bucket,
      region: target["settings"].s3_region,
      access_key_id: target["settings"].s3_access_key_id,
      secret_access_key: target["settings"].s3_secret_access_key
    }

    permanent_storage = if Hanami.env?(:test)
                          Shrine::Storage::FileSystem.new("spec/tmp/", prefix: "uploads")
                        else
                          Shrine::Storage::S3.new(**s3_options)
                        end

    Shrine.storages = {
      cache: Shrine::Storage::FileSystem.new("public", prefix: "uploads/cache"), # temporary
      store: permanent_storage
    }

    Shrine.plugin :entity
    Shrine.plugin :cached_attachment_data
    Shrine.plugin :restore_cached_data
    Shrine.plugin :form_assign
    Shrine.plugin :rack_file
    Shrine.plugin :validation_helpers
    Shrine.plugin :determine_mime_type

    register :shrine, Shrine
  end
end

We use quite some plugins, some are for caching the file, handling file through form, validations, mime_types. The most important one is entity which is a plugin for handling files being attaches to immutable objects.
This is a simple setup, that saves the files we add in specs to spec/tmp folder, and real files to s3. It also adds some plugins that we will use in the feature. We also need to add some settings:

#config/settings.rb
module Libus
  class Settings < Hanami::Settings
    setting :s3_bucket, constructor: Types::String
    setting :s3_region, constructor: Types::String
    setting :s3_access_key_id, constructor: Types::String
    setting :s3_secret_access_key, constructor: Types::String
  end
end

As per usual, we need a corresponding .env file with key-value pairs like S3_BUCKET=your_bucket_name etc.

As for the database structure, we need a table that needs a file attachment. Lets say we have a table books. In the migration (assuming it was the initial migration that created the table) we have:

column :image_data, :jsonb, null: true

And other columns. JSONB can also be text, but this setup is explained well in shrinerb. Compare this to active storage that adds two entire tables to your database, and stores all files there. Here you have data that you need, connected to relevant table.

Last part of the setup would be to create our Uploader.

#lib/libus/image_uploader.rb
require 'shrine'

module Libus
  class ImageUploader < Hanami.app["shrine"]
    TYPES = %w[image/jpeg image/png image/webp].freeze
    EXTENSIONS = %w[jpg jpeg png webp].freeze
    MAX_SIZE = 5 * 1024 * 1024
    MIN_SIZE = 1024
    Attacher.validate do
      validate_size MIN_SIZE..MAX_SIZE # 1kB..5MB
      validate_mime_type ImageUploader::TYPES
    end
  end
end

This is a class that will handle... uploading of our files. It describes what kind of files we accept, what is the size limit, and what is the minimum size. It is a simple example, but you can probably already see how you can expand it to fit your needs.

And just as a formality, we need some routes:

#slices/main/config/routes.rb
get '/books', to: 'books.index'
get '/books/:id/edit', to: 'books.edit'
patch '/books/:id', to: 'books.update'

As in the progress bar example, final feature will use a little bit of htmx, so bear in mind that this is also setup and affects the html and routing design.

Hanami code

So we have a spot in database, we have shrine setup. How do we implement it all? This framework is not described in shrine getting started guide, so we need to start from the ground up. I add some specs:

#spec/requests/image_upload_spec.rb
RSpec.describe 'ImageUploadSpec', :db, type: :request do
  context 'when photo is uploaded' do
    context "when user is logged in" do
      let!(:user) { factory[:user, name: "Guy", email: "[e-mail]"] }
      let!(:book) { factory[:book] }

      it 'changes the photo' do
        login_as user

        patch "/books/#{book.id}", { id: book.id, book: { image: Rack::Test::UploadedFile.new("spec/fixtures/image.png") } }

        expect(JSON.parse(rom.relations[:books].first.image_data)["id"]).to be_a(String)
        expect(last_response.status).to be(302)
      end
    end
  end
end

Very simple happy path test that checks that if we upload an image, it is saved in the database. It uses the simplest way to upload the file, and a simplest way to check the database. If image_data has an id in its jsonb structure, then it has a file attached.

Since we will have to modify our database, we probably will need to use a repository. It does not have a lot right now (only presenting relevant parts):

#slices/main/repositories/books.rb
module Main
  module Repositories
    class Books < Main::Repo[:books]
      struct_namespace Main::Entities
      commands :create, update: :by_pk, delete: :by_pk

      [...]
    end
  end
end

We do have an update command plugged in, but we can't simply update the image_data field it would skip all Shrine validations and checks, basically making the uploader obsolete.
Speaking of uploader, we need to add it to the entity, that was specified in the repository:

# slices/main/entities/book.rb
module Main
  module Entities
    class Book < ROM::Struct
      include Libus::ImageUploader::Attachment(:image)
      attr_writer :image_data
    end
  end
end

attr_writer is needed cause normally we don't change stuff on entities, but in this case, we need to change the image_data field, so we need to add a writer for it.
Of course that does not mean that the entity will gain the ability to change the data, it is still not its responsibility. But in case we use entity in a form, we can change the data in the form, and then pass it to the repository.

As for the include, it means that the Uploader will do its work on image related fields, meaning it expects image_data attribute on the connected object instance. If we provide that, we get access to all the shrine goods.

We should deal with the repository side and check if attaching the image does its job:

#spec/slices/main/repositories/books_spec.rb
context "#image_attach" do
let!(:dune) { factory[:book, title: "Dune", isbn_13: "9780441172719"] }
  it "succeeds" do
    expect(described_class.new.image_attach(
      dune.id,
      Rack::Test::UploadedFile.new("spec/fixtures/image.png")
    ).image).to be_a(Libus::ImageUploader::UploadedFile)
  end
end

A bit more precise spec, that assumes we will add a new method to the repo, that will attach an image and that the returned entity will have access to the image method entity and that also assumes that entity will have an image method (we did not define it, that is the shrine include doing this work for us).

As for the image_attach method, lets add it to the repository:

#slices/main/repositories/books.rb
def image_attach(id, image)
  book = books.by_pk(id).one!
  attacher = book.image_attacher
  attacher.form_assign({"image" => image})
  attacher.finalize
  self.update(book.id, attacher.column_values)
end

It uses the attacher we get access to from the entity, thanks to the include. Attacker gives us form_assign method that handles - you guessed it - uploads coming in form a form. We could use other methods, but for now we only expect files coming in from a form. After that we tell the attacher to finalize the upload, so move the file from cache, to persistent storage. After this we get access to column_values, which is what we should update the database record with.

Step by step, each easier than the last one, we receive the file (in cache), save it to the database, and the permanent storage (S3). Zero magic and only one column needed. 5 lines of code in the repo. We also have a way to check if the file is attached to the entity, and we can use it in the view.

View

Okey, so we got an index view, with list of books. On it, we can click edit on a row, and change the books picture (cover or whatever).
Part of that template could be:

#slices/main/templates/books/index.html.erb
<% books.each do |book| %>
  <tr>
    <th>
      <label>
        <input type="checkbox" class="checkbox" />
      </label>
    </th>
    <td>
      <div id=<%="picture-#{book.id}" %> class="flex">
        <div class="avatar">
          <div class="mask mask-squircle w-32 h-32">
            <%= book.avatar(:sm)%>
          </div>
        </div>
      </div>
    </td>
    <td>
      <div class="font-bold"><%= book.title %></div>
    </td>
    <td>
      <div class="font-bold"><%= book.category %></div>
    </td>
    <th>
      <div class="text-sm opacity-50"><%= book.author.name %></div>
    </th>
    <th>
      <button
        class="btn btn-sm btn-ghost"
        hx-get="/books/<%= book.id %>/edit"
        hx-target="<%="#picture-#{book.id}" %>"
        hx-swap="innerHTML"
        hx-trigger="click">
        Edit
      </button>
    </th>
  </tr>
<% end %>

A quick rundown of what htmx does here: it replaces the content (inner) of the div with the id picture-#{book.id} with the content of the response from the server, that is the edit form. It happens on button click.

Template with the form is nothing special

#slices/main/templates/books/edit.html.erb
<%= form_for :book, "/books/#{id}", method: :patch do |f| %>
  <%= f.file_field :image, hidden: true, value: book.image_data %>
  <%= f.file_field :image %>
  <button class="btn btn-primary" type="submit">Submit</button>
<% end %>

Here you can see why we need image_data attribute in the entity. We use it in the form, so it needs access to it.

Update

At this point, we basically need one line of code to upload the file.

#slices/main/actions/books/update.rb
module Main
  module Actions
    module Books
      class Update < Main::Action
        include Deps[books_repo: 'repositories.books']
        params do
          required(:id).filled(:integer)
          required(:book).schema do
            optional(:image).value(:hash)
          end
        end

        def handle(request, response)
          halt 422, {errors: request.params.errors}.to_json unless request.params.valid?

          books_repo.image_attach(request.params[:id], request.params[:book][:image][:tempfile])

          response.redirect_to "/books"
        end
      end
    end
  end
end

image_attach handles the shrine side of things that will validate the file against the rules we have set up. I skipped error handling for this, since it is no different that any other handling, specially since shrine also stores the initially uploaded file in the cache, and inserts it back in the form. Well it does not do so magically, we added the hidden field, that takes its value from image_data attribute.

Conclusion

ActiveStorage needs two tables. It breaks the usual style of rails, where every table is a model clearly defined in our app. Both tables have their corresponding objects, but they are hidden from out regular code, and not really changeable by us. We need to create an association to every model, and use the same tables for files on every model. The implementation is hidden from us, and file validation is not build in. Documentation is weird and the DSL is confusing.

On the other side, we get Hanami, with very clear and explicit way of setting providers and extensions, along with composable Shrine, that only needs one column and an include statement in an entity. Gives us a clear documentation and great DSL.

Hanami and Shrine are a great fit, and this integration was so easy and natural that I really appreciated how far a good design takes you in software development. Sure it might be obvious that good design and in general good code, provide good results, but it is rather rare to see two things integrate so well and seamless. Even if two pieces of software are well written and designed, there might be glitches and a lot of friction cause the design principles might differ. Not here though.

Also please do not take my criticism of ActiveStorage as a general slander of Rails or its contributors. I love Rails, and I owe my career to it and its many great contributors, I still love working with Rails. I just think that ActiveStorage is a bit of a misstep in the otherwise great framework. I am sure it will get better, but for now, I am sticking with Shrine.

Comments

[Paweł Świątkowski]

Thanks for another excellent article on Hanami. I'm comparing your setup with mine and I see you managed to avoid using shrine-rom gem, which I did not. On the other hand, I don't have an attr_writer on my entity ;) (not sure if this is thanks to the gem)

[Krzysztof]

At first I haven't looked at the gem too much to be honest, not sure how they get around the entity immutability, but will take a look out of curiosity.