DEV Community: benjaminolmsted

Canvas to S3 Buckets with Rails and React

benjaminolmsted — Thu, 30 Sep 2021 19:38:02 +0000

So you have a nice drawing on your beautiful HTML canvas. How can we persist our image? Save it to our database? Save it to our server? Well I'm using Heroku, so saving it to the server is a non starter (they use ephemeral file systems which will periodically wipe all the data that isn't in your slug). I also didn't want to put a blob or large binary file in my database. So I ended up storing the canvas contents as a PNG on Amazon's S3 cloud storage. I wanted to upload directly from the front-end so the basic flow of the app will be:

+User clicks save
+We send a request to the backend for a pre-signed URL that will let us upload to our S3 bucket
+We use an AWS gem to create the pre-signed URL and create a GET URL
+We send both the pre-signed URL and GET URL to the React front end
+Then we turn our canvas into a dataURL png and then into a blob png
+We POST that blob to the presigned URL and it gets uploaded to our S3 bucket
+On success, we POST the GET URL to our backend and create a resource that stores it

To implement client-side uploading to S3, there are 8 steps necessary to get programmatic uploading to S3 to work with a Rails backend and React front end. I'll outline them here, but I strongly recommend heading over to https://betterprogramming.pub/uploading-files-directly-to-aws-in-a-rails-react-app-9188f4eb6f7e for the walk through. I followed the steps closely, and got the process to work. In this example, an uploaded file was being saved, which is a slightly different process than saving from canvas. I'll go over those differences in detail. By the end we will be able to save our canvas contents to S3 and save the URL in our database for later access.

Steps 1-4
Setup an S3 Bucket to receive the images. This requires getting a FREE developer trial account, setting Bucket policies, Access Control Lists, CORS, and and creating a user with Access Key and a secret key for the bucket.

Step 5
Add the aws gem 'aws-sdk' ~3 NOTE: amazon has updated since the original blog post was written. Use version 3.

Add your keys to a .local_env.yml file and load it in applicaiton.rb

Step 6

Create an initializer

Step 7
Make a controller to handle the pre-signed URL creation

Step 8
Get the pre-signed url to the frontend and upload the image to S3, save the url where the image was posted.

This is where I diverged from the original blog.

saveImage

  async function saveImage(){        
        let canvas = ref.current
        let imageURL = canvas.toDataURL()
        const blob = dataURItoBlob(imageURL)
        const getURL = await uploadToAWS(blob, "postcards")
        const response = await fetch("/postcards",
                                    {method: "POST", 
                                    headers: {'Content-Type': 'application/json'}, 
                                    body: JSON.stringify({image_url: getURL, user_id: user.id})})
        const postcard = await response.json()
        setPostcards([...postcards, postcard]) 
}

here we get the data from our canvas into an image/png, base64 string via canvas.toDataURL().
Then we convert it into a blob with a handy function we found on stack overflow. I know that it works to convert the base64 string into a binary blob. How, well, I'm not sure. The code is below.
next we pass the blob to our uploadToAWS function, which does the magic and returns us a GET URL, the place the file is stored.
Then we POST to our backend, creating a postcard with the GET URL.
Lastly, we set state, adding the newly created postcard to our list of postcards.

The postcards_controller.rb

def create
        postcard = Postcard.create(postcard_params)
        render json: postcard
    end

    private
    def postcard_params
       params.permit(:image_url, :user_id)
    end
end

dataURItoBlob:

    function dataURItoBlob(dataURI) {
        var binary = atob(dataURI.split(',')[1]);
        var array = [];
        for(var i = 0; i < binary.length; i++) {
            array.push(binary.charCodeAt(i));
        }
        return new Blob([new Uint8Array(array)], {type: 'image/png'});
    }

here is the uploadToAWS function:

 const uploadToAWS = async(blob, directory) => {
        const  data  = await fetch(`/presign?filename=postcard&fileType=image/png&directory=${directory}`, 
                                    {method: "GET"})
        const json = await data.json();
        const { post_url, get_url } = json;
        const awsResp = await fetch(post_url, 
                                    {method: 'PUT', 
                                    headers: {"Content-Type": 'image/png','acl': 'public-read'}, 
                                    body: blob})
        return get_url;
  }

First we get out pre-signed url from the backend, along with the location where we will upload the file to.
Then we PUT the blob to the pre-signed url post_url
The we return the location of the file in get_url

And just like that, our pretty pictures are in the S3 bucket, waiting to be served, and we have their location saved to our database.

Minting NFTs

benjaminolmsted — Wed, 22 Sep 2021 20:58:32 +0000

A brief look at an evolving ecosystem.

An NFT is a non-fungible token, living somewhere on some blockchain. They can be linked to any digital asset like a piece of art, which can be a still image, movie, 3D model, or even an interactive web app. The marketplaces that have sprung up differ slightly in what formats they accept, how much it costs to mint, and what blockchain they run on.

A blockchain is a growing list of records, called blocks, that are linked together using cryptography. Each block contains a cryptographic hash of the previous block, a timestamp, and transaction data As blocks each contain information about the block previous to it, they form a chain, with each additional block reinforcing the ones before it. Therefore, blockchains are resistant to modification of their data because once recorded, the data in any given block cannot be altered retroactively without altering all subsequent blocks. -wikipedia

There are a bunch of NFT marketplaces:

OpenSea

https://www.hicetnunc.xyz/

https://beyondnft.io/

https://rarible.com/

Uploads and Factories, or the minting magic.

There are a couple ways to mint an NFT. One is through direct upload. All the platforms seem to accept this way of doing things. Another way is via a Factory. A factory allows a collector to set some parameters on the work before minting it.

Here is the flow for adding a work to https://www.hicetnunc.xyz/

Upload your NFT. I chose an animated gif.
Approved in my Tezos wallet.
Go to the newly minted creation!
Swap it to set a sale price.

See it live!

Factory

This is a factory on beyondNFT. They have an in-browser coding setup that allows users to create interactive NFTs.

Collections, Collaborations, and Clubs

https://openprocessing.org/crayon/

https://www.theblocksofart.com/

https://boredapeyachtclub.com/

https://feralfile.com/

Setting up an API with Rails

benjaminolmsted — Fri, 03 Sep 2021 19:46:52 +0000

In this blog, we go over the steps for setting up a Rails based API.

installation
CORS
generate resource
create migration
add seed data
add actions to our controller
testing in Postman
some validations

1

run rails new this-is-a-blog --api

This sets up the project for use as an API.

2

enable CORS (cross-origin resource sharing)
In the Gemfile, uncomment the gem 'rack-cors'

Then head over to config/initializers/cors.rb and uncomment lines 8-16 and change origins 'example.com' to origins '*' to allow requests from anywhere. If the API is meant to be consumed by a specific domain, put that domain here. For development purposes, '*' is fine.

bundle install

3

we generate our first resource with rails g resource Post

This generates a post.rb model, a posts_controller.rb controller, a CreatePosts migration, and adds a resources: posts to our routes.rb file.

4

we create our migration in db/migrate/{local_time}create_posts.rb

class CreatePosts < ActiveRecord::Migration[6.1]
  def change
    create_table :posts do |t|
      t.string :title
      t.text :body
      t.string :image_url
      t.timestamps
    end
  end
end

we could have had this step done automatically if we had run our generation like so: rails g resource Post title body image_url

and migrate with rails db:migrate

5

we add some seed data to db/seeds.rb

Post.create(title: "Editor Basics", body: "Use Markdown to write and format posts. You can use Liquid tags to add rich content such as Tweets, YouTube videos, etc.
    In addition to images for the post's content, you can also drag and drop a cover image")
Post.create(title: "Setting up an API with Rails", body:"First, run `rails new this-is-a-blog --api`") 
Post.create(title: "Writing a Great Post Title", body: "Think of your post title as a super short (but compelling!) description — like an overview of the actual post in one short sentence.
Use keywords where appropriate to help ensure people can find your post by search.")

and run rails db:seed

6

we head over to app/controllers/posts_controller.rb and lay out our actions

class PostsController < ApplicationController
rescue_from ActiveRecord::RecordNotFound, with: :record_not_found_response
rescue_from ActiveRecord::RecordInvalid, with: :record_invalid_response

    def index
        posts = Post.all
        render json: posts, status: :ok
    end

    def show
        post = find_post
        render json: post, status: :ok
    end

    def update
        post = find_post
        post.update!(post_params)
        render json: post, status: :accepted
    end

    def create
        post = Post.create!(post_params)
        render json: post, status: :created
    end

    def destroy
        post = find_post
        post.destroy
        head :no_content
    end

    private
    def find_post
        Post.find(params[:id])
    end

    def post_params
        params.permit [:title, :body, :image_url]
    end

    def record_not_found_response(e)
        render json: {error: e.to_s}, status: :not_found
    end

    def record_invalid_response(invalid)
        render json: {errors: invalid.record.errors.full_messages}, status: :unprocessable_entity
    end
end

7

now it's time to check our work using Postman

Fire up your server with rails s

Index is working!

Our Show route works!

So does the create route.

our patch updates.

Our delete deletes.
Our happy routes work! Excellent. Now let's check our error handling.

When we try to find an :id that does not exist, we get our helpful error!

Excellent! We made an API in rails.

8

now, let's put in some validations on our model.

class Post < ApplicationRecord
    validates :title, presence: true, length: {minimum: 2}
    validates :body, presence: true
end

Now let's see what happens if we try to post something without a body!

We cannot, thanks to the validations. We get that error thanks to our rescue_from ActiveRecord::RecordInvalid and our use of create! and update!, which will throw an ActiveRecord::RecordInvalid error if the request fails any validations.

The very basics of setting up an API in Rails, complete.

Resources:
https://guides.rubyonrails.org/active_record_validations.html

https://guides.rubyonrails.org/api_app.html

https://dev.to/lisahjung/beginner-s-guide-to-creating-an-api-from-scratch-using-rails-2eie

Ruby Regex

benjaminolmsted — Fri, 20 Aug 2021 17:54:21 +0000

I sort of blackout when I get hit with a regex string. this seems suboptimal. This is me learning about Regex. The Rubular tool is indispensable https://rubular.com/

Regular Expressions are used to match patterns in text. Why might we want to do this?
Data Validation - is a string a valid email address or phone-number?

(?:[a-z0-9!#$%&'+/=?^_{|}~-]+(?:\\.[a-z0-9!#$%&'*+/=?^_{|}~-]+)|"(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])")@(?:(?:a-z0-9?\.)+a-z0-9?|\[(?:(?:(2(5[0-5]|[0-4][0-9])|1[0-9][0-9]|[1-9]?[0-9]))\.){3}(?:(2(5[0-5]|[0-4][0-9])|1[0-9][0-9]|[1-9]?[0-9])|[a-z0-9-][a-z0-9]:(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21-\x5a\x53-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])+)\])

(Well, actually, email validation should be done through a confirmation link with a token.)

Web scrapping

Any text parsing needs like syntax highlighting, data wrangling, or search

Constructing Regular Expressions

They can be made several ways. With literals:

/ pattern / or %r{pattern}

They can also be constructed using [Regex.new](http://regex.new)with a string and Regex.union with a list of strings or an array of strings.

As for the contents of the pattern, the following table is an excellent starting place. I couldn't make it cleaner, so here it is in it's entirety.

Using Regular Expressions

=~ is Ruby's basic pattern-matching operator.

It will return the first index of a match in the string, nil if no match is present.

#match will return a MatchData object

In the String class, there are several methods that make use of regular expressions

#gsub can be used to substitute a string for all occurrences given regular expression.

#sub substitutes the first occurrence of the supplied regex

#partition splits the string into three: the part before the match, the match, and the part after the match

#scan produces an array of the matches or passes them to a block

#split splits a string on the given pattern

What can we put in the pattern?

Characters!

Individual characters represent themselves literally

/abc/ will match 'a' followed by 'b' followed by 'c', exactly, but not something like "abdc"

/[abc]/ will match a or b or c

Here we match one of [aeiou] followed by a single t

like all meta-characters, if we want to use the brackets in our literal pattern, we have to escape them with back-slashes [.

here we match the footnote markers from wikipedia by using the special character \d, which matches any digit

In addition to literal characters, there are many special characters we can use to build our patterns.

/\w/ - A word character ([a-zA-Z0-9_])

/\W/ - A non-word character ([^a-zA-Z0-9_])

/\d/ - A digit character ([0-9])

/\D/ - A non-digit character ([^0-9])

/\h/ - A hexdigit character ([0-9a-fA-F])

/\H/ - A non-hexdigit character ([^0-9a-fA-F])

/\s/ - A whitespace character: /[ \t\r\n\f\v]/

/\S/ - A non-whitespace character: /[^ \t\r\n\f\v]/

A nice convention we see here is that by upcasing the letter, we get the complimentary set of characters.

We can also specify the number of times that a character occurs using quantifiers.

* - Zero or more times

+ - One or more times

? - Zero or one times (optional)

{n} - Exactly n times

{n,} - n or more times

{,m} - m or less times

{n, m} - At least n and at most m times

Here we find all double vowels.

Parentheses can be used to create capture groups, which allow us to access parts of the match later in the pattern, and also after we match, by using the match variables $1, $2, $3...

There is a lot to parentheses in regex. I'll leave it here for now.

References

https://en.wikipedia.org/wiki/Regular_expression

https://ruby-doc.org/core-2.4.1/Regexp.html

https://ruby-doc.org/core-2.4.0/MatchData.html

Procedural Animation with Canvas, random walks

benjaminolmsted — Fri, 23 Jul 2021 17:18:25 +0000

HTML5 comes with Canvas, a beautiful blank canvas for us to paint on.
It can do a lot. I'll be going through how to create a simple procedural animation! It shouldn't take long to get a random walker moving around the screen. After we get our pixel friend moving, we'll play with it's color. Then we'll get a few walkers moving on the canvas at the same time. Finally, we'll connect the dots to get our walkers painting an abstract.

Our final walkers will draw something like this:

But first, we need a canvas!

<canvas id='mainCanvas'>Displayed if canvas isn't supported.</canvas>

I like the canvas to fill our whole view:

canvas.width = window.innerWidth
canvas.height = window.innerHeight

Once we have our canvas, we need it's context, the object we use to manipulate our canvas.

ctx = canvas.getRenderContext('2d')

That's all we need to start rendering shapes to our canvas.

let rectWidth = 5
let rectHeight = 5
ctx.fillStyle = 'black'
ctx.rect( canvas.width/2, canvas.height/2, rectWidth, rectHeight)
ctx.fill()

open on codepen
Now we have a point in the center of the page!

To animate the point, we need two things.

An animation loop and some rendering logic for the point that will be called each time the loop runs.

Our rendering logic will be ultra simple for now: at each step, we will move the x and y portions of our point by a random amount.
Then we will draw it to the screen.

function render(){
  point.x += (Math.random()-.5)*5
  point.y += (Math.random() -.5)*5
  ctx.rect( point.x, point.y, 5, 5)
  ctx.fill()
  window.requestAnimationFrame(render);
}

Then we get our animation loop from window.requestAnimationFrame(callback), which fires when the window is ready to draw a new frame.
We then give it our callback function that renders the new frame. (If animation speed is of concern, we should use the time passed to the callback to keep the animation steady across monitors with different refresh rates)

We have our animation running, just like that. Feel free to play with it on codepen!

Now we are going to play a little bit with color.

ctx.fillStyle = `rgba(${time%255}, ${time%50}, ${time%127}, 1)`

The context fillStyle can understand any valid CSS color value, including named strings, rgba() values, as well as many others.
We use the passed time along with the modulo operator to create a cycling color pallet. The red channel varies from 0-255, the green from 0 to 50, and the blue from 0 to 127.

Next, we'll add a few random walkers to our screen. Why? Because we can!

First we setup our walkers.

let points = []
for(let i = 0; i < numPoints; i++){
  let point = {x: canvas.width/numPoints*i, 
             y: canvas.height/2}
  points.push(point)
}

Then we update each of the walkers in our render function.

points.forEach( (point, i) =>{ 
    ctx.beginPath()
    point.x += (Math.random()-.5)*5
    point.y += (Math.random() -.5)*5
    ctx.rect( point.x, point.y, 5, 5)
    ctx.fill()
  })

What happens when we connect all the walkers with a colored line?
Let's find out!

with 'hue' compositing

Using the center point as the connected point, dodge compositing, smaller walkers, and a bigger y-direction jump.

Same setup, longer run

Here is the codepen that generated it