Uploading and Validating Images with Crystal and Lucky on Heroku

Today we're going to be uploading images with Lucky and crystal! To demo this I'm going to make an app that allows uploading images to a gallery that is tied to your ip address. Since we're going to host this app on heroku, we can use heroku's X-FORWARDED_FOR header to get the user's ip address.

Note that because of proxys and the potential for ip spoofing, this is not a secure method of restricting user access and I won't recommend using it for any important data.

Finished Code

To see the finished code and run it locally you can clone the repo and checkout the image-uploads branch.

git clone [e-mail]:mikeeus/lucky_demo.git
git checkout image-uploads

cd image-uploads
bin/setup
lucky db.create && lucky db.migrate

And you can run the specs to see the beautiful green result :).

crystal spec spec/flows/images_spec.cr

Image Table

First lets create the Image table.

lucky gen.migration CreateImages

We'll add the following columns to hold the filename, ip address of the owner and we'll even record the number of times the image is viewed by users.

class Image::V20180728000000 < LuckyMigrator::Migration::V1
  def migrate
    create :images do
      add filename : String
      add owner_ip : String
      add views : Int32
    end

    execute "CREATE INDEX owner_ip_index ON images (owner_ip);"
    execute "CREATE UNIQUE INDEX filename_index ON images (filename);"  
  end

  def rollback
    drop :images
  end
end

We'll also add a unique index on the filename and a normal index on the owner_ip column so we can quickly get collections of images based on it.

Specs

When allowing uploads to our app we'll want to restrict the files by type and possible dimensions. We'll create specs to check this for us. Unfortunately Crystal doesn't give us information on an image's dimensions our of the box, so we'll later we'll use crymagic to get this info for us.

The limits we'll put on our uploads are the following:

• formats: JPG, GIF, PNG
• max dimensions: 1000x1000
• max size: 250kb

I've added some images to our assets folder that break each of these rules as well as one image that is perfect.

ALSO: I got these images from this amazing site: africandigitalart.com, which I recommend checking out.

public/
  assets/
    images/
      test/
        perfect_960x981_56kb.jpg
        too_big_900x900_256kb.jpg
        too_tall_1001x1023_95kb.jpg
        wrong_format_240x245_235kb.bmp

Next we'll create an ImageBox in case we need to instantiate Images in our tests.

# spec/support/boxes/image_box.cr
class ImageBox < LuckyRecord::Box
  def initialize
    filename "perfect_960x981_56kb.jpg"
    owner_ip "0.0.0.0"
    views 1
  end
end

Lucky Flow

Lucky uses the concept of Flows which are classes that encapsulate the behavior of your browser tests. We'll create one now that uploads an image on our homepage and has two methods for checking if it succeeded or not.

We can simulate uploading a file by adding the file's full path to the file input of the form. Then theclick "@upload-image" method will look for an element with [flow_id=upload-image] tag on the page and click it.

# spec/support/flows/images_flow.cr
class ImagesFlow < BaseFlow
  def upload_image(filepath)
    visit Home::Index

    fill_form ImageForm,
      image: File.expand_path(filepath)
    click "@upload-image"
  end

  def image_should_be_created(filepath)
    image = find_image_by_filename?(File.basename(filepath))
    image.should_not be_nil
  end

  def image_should_not_be_created(filepath)
    image = find_image_by_filename?(File.basename(filepath))
    image.should be_nil
  end

  private def find_image_by_filename?(filename)
    ImageQuery.new.filename.ilike("%#{filename}%").first?
  end
end

Now we can use this flow and our test images to write our specs. Crystal has first class support for specs and we can see that by how simple it is to write them. We use Spec.after_each to clear the images with a delete! method that will also delete the underlying file after every spec.

# specs/flows/images_spec.cr
require "../spec_helper"

describe "Images flow" do
  Spec.after_each do
    ImageQuery.new.map(&.delete!)
  end

  describe "uploading" do
    it "works with valid image" do
      flow = ImagesFlow.new

      flow.upload_image(valid_image_path)
      flow.image_should_be_created(valid_image_path)
    end

    it "doesnt work with image above 250kb" do
      flow = ImagesFlow.new

      flow.upload_image(too_big_image_path)
      flow.image_should_not_be_created(too_big_image_path)
    end

    it "doesnt work with dimensions over 1000x1000" do
      flow = ImagesFlow.new

      flow.upload_image(too_tall_image_path)
      flow.image_should_not_be_created(too_tall_image_path)
    end

    it "doesnt work with image of the wrong format" do
      flow = ImagesFlow.new

      flow.upload_image(wrong_format_image_path)
      flow.image_should_not_be_created(wrong_format_image_path)
    end
  end
end

private def valid_image_path
  "public/assets/images/test/perfect_960x981_56kb.jpg"
end

private def too_tall_image_path
  "public/assets/images/test/too_tall_1001x1023_95kb.jpg"
end

private def too_big_image_path
  "public/assets/images/test/too_big_900x900_256kb.jpg"
end

private def wrong_format_image_path
  "public/assets/images/test/wrong_format_240x245_235kb.bmp"
end

Running these specs will cause them to fail since we haven't implemented anything. Let's now build out our models, actions and pages to make them work.

Image Model

We'll need to persist references to our images, their owner's ip and number of views to the database. So let's generate a model to do that.

lucky gen.model Image

And we can fill out the Image model with its columns and a some helper methods to build the path, url and handle deletion. The images will be saved at public/assets/images/..., and will be available publicly at at www.example.com/assets/images/.... We'll also add a case for test images that will be stored under the public/assets/images/test/ directory.

# src/models/image.cr
class Image < BaseModel
  table :images do
    column filename : String
    column owner_ip : String
    column views : Int32
  end

  def url
    "#{Lucky::RouteHelper.settings.base_uri}#{path}"
  end

  def path
    if Lucky::Env.test?
      "/assets/images/test/#{self.filename}"
    else
      "/assets/images/#{self.filename}"
    end
  end

  def full_path
    "public#{path}"
  end

  def delete!
    File.delete(full_path)
    delete
  end
end

Next we can fill out our ImageForm. Forms in Lucky are responsible for creating and updating models. We use fillable to declare which columns we'll be updating, and we'll declare a virtual field image to hold our uploaded image until we can save it. We'll also add needs file and needs ip because we'll be passing these in when instantiating the form.

uuid is used to make sure we have unique filenames and make it almost impossible for someone to view the image without the filename.

We put all of this together in the prepare method which saves the image and sets the columns. It currently doesn't do any validations but we'll get to that later.

require "uuid"

class ImageForm < Image::BaseForm
  fillable filename
  fillable views
  fillable owner_ip

  virtual image : String

  needs file : Lucky::UploadedFile, on: :create
  needs ip : String, on: :create

  getter new_filename

  def prepare
    save_image
    views.value = 1
    filename.value = new_filename
    owner_ip.value = ip
  end

  private def uploaded
    file.not_nil!
  end

  private def save_image
    File.write(save_path, File.read(uploaded.tempfile.path))
  end

  private def new_filename
    @new_filename ||= "#{UUID.random}_#{uploaded.metadata.filename}"
  end

  private def image_path
    if Lucky::Env.test?
      "assets/images/test/" + new_filename
    else
      "assets/images/" + new_filename
    end
  end

  private def save_path
    "public/" + image_path
  end
end

Now we need to create the UI to allow uploads and the actions to save the forms.

Home Page

Currently our app displays Lucky's default homepage. We'll create a new Home page that holds our form and allow us to upload files. Let's generate the page.

lucky gen.page Home::IndexPage

Then we'll add a form that has enctype: "multipart/form-data" and posts to Images::Create which will handle creating our Image. We add needs form : ImageForm to tell the action that renders this page to pass in a new form. We'll also render any errors in a list below the input.

class Home::IndexPage < GuestLayout
  needs form : ImageForm

  def content
    render_form(@form)
  end

  private def render_form(f)
    form_for Images::Create, enctype: "multipart/form-data" do
      text_input f.image, type: "file", flow_id: "file-input"      

      ul do
        f.image.errors.each do |err|
          li "Image #{err}", class: "error"
        end
      end

      submit "Upload Image", flow_id: "upload-image"
    end
  end
end

And let's change our Home::Index action to show our index page rather than Lucky's welcome page.

# src/actions/home/index.cr
class Home::Index < BrowserAction
  include Auth::SkipRequireSignIn
  unexpose current_user

  get "/" do
    if current_user?
      redirect Me::Show
    else
      render Home::IndexPage
    end
  end
end

Get current_ip in Actions

We won't be using current_user for authentication, instead we need to get the ip address of the request. When our app is on heroku we can use the X-FORWARDED-FOR header which is set automatically. Locally we'll just set it to local.

We'll add these methods in the BrowserAction. Since our other actions inherit from it class Home::Index < BrowserAction, it will make these methods available for us.

# src/actions/browser_action.cr
abstract class BrowserAction < Lucky::Action
  ...
  def current_ip
    current_ip?.not_nil!
  end

  private def current_ip?
    if Lucky::Env.production?
      context.request.headers["X-FORWARDED-FOR"]?
    else
      "local"
    end
  end
  ...
end

Images Create Action

Now we need an action to handle the image creation after we submit the form on the home page. Let's generate one with:

lucky gen.action.browser Images::Create

For more information on how actions work, you can check out Lucky's guides.

This action will get the file from the params which will be in the form { "image": { "image": "file is here" }}. If it's not nil we'll pass the file as well as the current_ip to the ImageForm which will validate and save our new Image.

To check that our file exists we'll make sure its not nil and that the filename exists.

# src/actions/images/create.cr
class Images::Create < BrowserAction
  include Auth::SkipRequireSignIn
  unexpose current_user

  route do # lucky expands this to: post "/images"
    file = params.nested_file?(:image)["image"]?

    if is_invalid(file)
      flash.danger = "Please select a file to upload"
      redirect to: Home::Index
    else
      ImageForm.create(file: file.not_nil!, ip: current_ip) do |form, image|
        if image
          flash.success = "Image successfuly uploaded from #{current_ip}!"
          redirect to: Home::Index
        else
          flash.danger = "Image upload failed"
          render Home::IndexPage, form: form
        end
      end
    end
  end

  private def is_invalid(file)
    file.nil? || file.metadata.filename.nil? || file.not_nil!.metadata.filename.not_nil!.empty?    
  end
end

And voila! Our app can now handle image uploads.

If we run the specs with lucky spec spec/flows/images_spec.cr we'll see that our first spec that checks valid images will pass, but since we haven't implemented image validations the rest will fail.

Validations

In order to check the images' file size, type and dimensions we're going to use a little gem of a shard called crymagick. It requires having ImageMagick installed which luckily for us is present on Heroku by default. If it's not installed on your local machine you can get it from the official site here.

Lets install the shard by adding it to the bottom of our dependencies in shard.yml and running shards.

# shard.yml
...
dependencies:
  ...
  crymagick:
    github: imdrasil/crymagick

Now we can use it in our ImageForm to validate our images. We add three methods validate_is_correct_size, validate_is_correct_dimensions and validate_is_correct_type that will use CryMagick::Image to check the file's type, size and dimensions. If there are no errors, we move on to saving the file and setting the Image's columns.

require "uuid"
require "crymagick"

class ImageForm < Image::BaseForm
  ...
  getter crymagick_image : CryMagick::Image?

  def prepare
    validate_is_correct_size
    validate_is_correct_dimensions
    validate_is_correct_type

    if errors.empty? # save if validations pass
      save_image

      views.value = 1
      filename.value = new_filename
      owner_ip.value = ip
    end
  end

  private def validate_is_correct_type
    ext = crymagick_image.type

    unless Image::VALID_FORMATS.includes? "#{ext}".downcase
      image.add_error "type should be jpg, jpeg, gif or png but was #{ext}"
    end
  end

  private def validate_is_correct_size
    size = crymagick_image.size # returns size in bytes

    if size > 250_000 # 250kb limit
      image.add_error "size should be less than 250kb but was #{size / 1000}kb"
    end
  end

  private def validate_is_correct_dimensions
  dimensions = crymagick_image.dimensions # returns (width, height)

    if dimensions.first > 1000
      image.add_error "width should be less than 1000px, but was #{dimensions.first}px"
    end

    if dimensions.last > 1000
      image.add_error "height should be less than 1000px, but was #{dimensions.last}px"
    end
  end

  private def crymagick_image # To avoid opening the file multiple times
    @crymagick_image ||= CryMagick::Image.open(uploaded.tempfile.path)
  end
  ...
end

Now if we run the specs we'll see that they all pass! Hurray!

All thats left now is to add support for deleting and viewing our images.

Displaying and Deleting Images

What we want is to display our images on the home page as a gallery. Each image should have a button to delete and should display it's url.

Let's begin with a spec that visits the homepage and checks for images on the screen, and another one that clicks the delete button and checks that the image is deleted.

# specs/flows/images_spec.cr
require "../spec_helper"

describe "Images flow" do
  ...
  describe "displays" do
    it "own images" do
      flow = ImagesFlow.new

      owned = ImageBox.new.owner_ip("local").create
      not_owned = ImageBox.new.owner_ip("not-owned").create

      flow.homepage_should_display_image(owned.id)
      flow.homepage_should_not_display_image(not_owned.id)
    end
  end

  describe "deleting" do
    it "is allowed for owner" do
      flow = ImagesFlow.new

      flow.upload_image(valid_image_path)

      image = ImageQuery.new.first

      flow.delete_image_from_homepage(image.id)
      flow.image_should_not_exist(image.id)
    end

    it "is not allowed for other ip addresses" do
      flow = ImagesFlow.new
      not_owned = ImageBox.new.owner_ip("not-local").create

      flow.delete_image_from_action(not_owned.id)
      flow.image_should_exist(not_owned.id)
    end
  end
end
...

And lets add the flows that will visit the homepage, check for images, check for images in the database, and delete images by pressing buttons or visiting the actions directly.

class ImagesFlow < BaseFlow
  ...
  def homepage_should_display_image(id)
    visit Home::Index
    image(id).should be_on_page    
  end

  def homepage_should_not_display_image(id)
    visit Home::Index
    image(id).should_not be_on_page    
  end

  def delete_image_from_homepage(id)
    visit Home::Index
    click "@delete-image-#{id}"
  end

  def delete_image_from_action(id)
    visit Images::Delete.with(id: id)
  end

  def image_should_exist(id)
    ImageQuery.find(id).should_not be_nil
  end

  def image_should_not_exist(id)
    ImageQuery.new.id(id).first?.should be_nil    
  end

  ...

  private def image(id)
    el("@image-#{id}")
  end

Our tests will be failing now, so lets add support for displaying images by updating our Home::IndexPage. We'll require that the page is rendered with an images prop that is an ImageQuery. Then we'll use the images in a new gallery method that renders each image including links to delete it and a url to display it.

class Home::IndexPage < GuestLayout
  needs form : ImageForm
  needs images : ImageQuery # ADD THIS!

  def content
    div class: "homepage-container" do
      render_form(@form)

      gallery # add gallery erhere
    end
  end

  private def gallery # define it here
    h2 "Image Gallery"

    ul class: "image-gallery" do
      @images.map do |image|
        li class: "image", flow_id: "image-#{image.id}" do
          div class: "picture", style: "background-image: url(#{image.path});" do
            div "Views: #{image.views}", class: "views"
          end

          link to: Images::Delete.with(image.id), flow_id: "delete-image-#{image.id}" do
            img src: asset("images/x.png")
          end

          div image.url, class: "image-url",  flow_id: "image-url-#{image.id}"
        end
      end
    end
  end
  ...
end

I've also added styles to src/css/app.scss which I won't include in this article.

In order for this to work we need to update our actions that render the Home::IndexPage so that they pass in the images.

# src/actions/home/index.cr
class Home::Index < BrowserAction
  ...

  get "/" do
    if current_user?
      redirect Me::Show
    else
      images = ImageQuery.new.owner_ip(current_ip)
      render Home::IndexPage, form: ImageForm.new, images: images # pass it in here
    end
  end
end

And in our Images::Create action.

# src/actions/images/create.cr
class Images::Create < BrowserAction
  ...
  route do
    if is_invalid(file)
      ...
    else
      ImageForm.create(file: file.not_nil!, ip: current_ip) do |form, image|
        if image
          ...
        else
          ...
          images = ImageQuery.new.owner_ip(current_ip)
          render Home::IndexPage, form: form, images: images # pass it in here
        end
      end
    end
  end
  ...
end

All set! The Home::IndexPage won't complain about not having images passed in. But it will complain about a link to Images::Delete which hasn't been implemented. So let's do that now.

lucky gen.action.browser Images::Delete

The Images::Delete action should check if the current_ip matches the Image's owner_ip and if so call delete!.

# src/actions/images/delete.cr
class Images::Delete < BrowserAction
  include Auth::SkipRequireSignIn
  unexpose current_user

  route do # expands to: delete "/images/:id"
    image = ImageQuery.find(id)

    if image.owner_ip == current_ip
      image.delete!
      flash.success = "Image succesfully deleted!"
      redirect to: Home::Index
    else
      flash.danger = "You are not authorized to delete this image"
      redirect to: Home::Index
    end
  end
end

Now run the tests and... Boom! Green.

Show Single Image

The last thing we'll implement is a show page for each image that updates the number of views. Let's generate the action, page and a form to update images for us.

lucky gen.action.browser Images::Show
lucky gen.page Images::ShowPage

touch src/forms/image_views_form.cr # no generater for forms atm

The form will be simple and only be used for incrementing the value. It can be used like this: ImageViewsForm.update!(image).

# src/forms/image_views_form.cr
class ImageViewsForm < Image::BaseForm
  fillable views
  fillable filename
  fillable owner_ip

  def prepare
    views.value = views.value.not_nil! + 1
  end
end 

For our action we'll use a custom route so that our route parameter is available as filename instead of id. Then we check that it exists and increment the views and render the page, otherwise we redirect to the Home::Index action.

# src/actions/images/show.cr
class Images::Show < BrowserAction
  include Auth::SkipRequireSignIn
  unexpose current_user

  get "/images/:filename" do
    image = ImageQuery.new.filename(filename).first?
    if image.nil?
      flash.danger = "Image with filename: #{filename} not found"
      redirect to: Home::Index
    else
      ImageViewsForm.update!(image)
      render Images::ShowPage, image: image
    end
  end
end

The show page will be very simple. We display the filename, the views and the image using minimal style to keep everything centered while allowing the image to stretch to its full size.

# src/pages/images/show_page.cr
class Images::ShowPage < GuestLayout
  needs image : Image

  def content
    div style: "text-align: center;" do
      h1 @image.filename
      h2 "Views: #{@image.views}"
      img src: @image.path, style: "max-width: 100%; height: auto;"
    end
  end
end

To finish off we'll make the image displayed on the home page link to the Images::ShowPage.

class Home::IndexPage < GuestLayout
  ...
    ul class: "image-gallery" do
      @images.map do |image|
        li class: "image", flow_id: "image-#{image.id}" do
          link to: Images::Show.with(image.filename), # Changed this to link: ..
               class: "picture",
               style: "background-image: url(#{image.path});" do
            div "Views: #{image.views}", class: "views"
          end
          ...
        end
      end
    end
  ...
end

And we're done! The tests should all be green and the app working as expected.

Join Us

I hope you enjoyed this tutorial and found it useful. Join us on the Lucky gitter channel to stay up to date on the framework or checkout the docs for more information on how to bring your app idea to life with Lucky.