Creating a JSON API with Athena & Granite

UPDATE: April 22, 2019 for Athena version 0.6.0
UPDATE: November 24, 2019 for Athena version 0.7.0 and Crystal 0.31.1
UPDATE: February 7, 2020 for Athena version 0.8.0
UPDATE: June 10, 2020 for Athena version 0.9.0
UPDATE: July 11, 2020 for Athena version 0.10.0

Athena

Intro

A few months ago (over a year ago at this point) I set out to create a new web framework, but with a few key differences. I wanted something that would allow for a route's action to be easily documented, tested, and flexible. I also didn't want to have to deal with the boilerplate of converting route/query/body params into their expected types manually all the time in every route. Finally, I wanted to take advantage of Crystal's annotations to provide a simple yet flexible DSL for defining routes.

Taking the all of the frameworks I have experienced into consideration, with big help from Symfony. The outcome of this idea was Athena.

Now, I wanted to write a blog post showing how a JSON API with Athena would look like in an actual app, outside of general documentation.

Tutorial

This tutorial will not cover any front end work (UI/UX). It will just assume that the requests coming to the API are coming from a frontend JS framework or something. Requests are JSON, so it is pretty framework agnostic.

I'll be taking a pretty slow approach, as to make this tutorial applicable to both new crystalers as well as veterans.

Requirements

• Crystal installed on your machine. (Latest version as of writing is 0.35.1)
• Your HTTP client of preference. I'll just be using cURL
• Your IDE/editor of preference.
• Your DB of preference. I will be using Postgres.
  • (Optional) Docker. Is what I will be running my DB with.

Agenda

Add the ability to:

• Register/Login
  • Validate users
• Create/Read/Update/Delete articles
  • Validate articles

Scaffolding the blog

We can utilize the crystal binary to scaffold out our application. This will create a new directory with the given name, with the required files for a crystal app; including the basic directory structure, a shard.yml, and a .gitignore, all auto generated for us. I will go ahead and create this in my home directory, then cd into the newly created directory; ready for the next steps.

$ cd ~/
$ crystal init app blog
$ cd ./blog

Dependencies

I will be using Granite as our ORM of choice to pair with Athena. Start off by adding the following to your shard.yml file.

I will also be requiring the jwt shard to generate JWTs to use as our authentication method of choice.

NOTE: I am using Postgres, and as such am installing the PG shard for use with Granite. If you are using another DB adapter, you will need to install that shard instead.

dependencies:
  granite:
    github: amberframework/granite
    version: 0.21.1
  pg:
    github: will/crystal-pg
    version: 0.21.1
  athena:
    github: athena-framework/athena
    version: 0.10.0
  jwt:
    github: crystal-community/jwt
    version: 1.4.2
  assert:
    github: blacksmoke16/assert
    version: 0.2.0

Then install the required dependencies:

$ shards install

Defining Our Models & Controllers

Our blog will have two models, and two database tables:

1. User - Stores users that have registered with our blog
2. Article - A blog post authored by a user.

For the purpose of this tutorial, I will just be executing the raw SQL in Postgres to create the tables. There are some migration shards out there that can automate this; could be a future iteration.

First lets create a new schema to hold our tables, as well as give our DB user access to that database.

I will be running my PG database using docker, with the following compose file:

version: '3.1'
services:
  pg:
    image: postgres:11.2-alpine
    container_name: pg
    ports:
      - "5432:5432"
    environment:
      - POSTGRES_USER=blog_user
      - POSTGRES_PASSWORD=mYAw3s0meB!log
      - POSTGRES_DB=blog
    volumes:
      - pg-data:/var/lib/postgresql/data

volumes:
  pg-data:

CREATE SCHEMA "blog";
ALTER ROLE "blog_user" SET SEARCH_PATH TO "blog";

NOTE: Using Docker is optional, as long as you have a DB to connect to you'll be fine.

Now that our dependencies are installed, we need to require them, as well as setup our DB connection in our blog.cr file. It should look something like:

# Register an adapter to connect to our DB
Granite::Connections << Granite::Adapter::Pg.new(name: "my_blog", url: "postgres://blog_user:mYAw3s0meB!log@localhost:5432/blog?currentSchema=blog")

# Require some standard library things we'll need
require "crypto/bcrypt/password"

# Require our ORM and DB adapter
require "granite"
require "granite/adapter/pg"

# Require Athena
require "athena"

# This will eventually be replaced by Athena's validation component
require "assert"

# Require JWT shard
require "jwt"

module Blog
  VERSION = "0.10.0"

  # Runs the HTTP server with the default settings
  ART.run
end

User Model

Next lets think about what columns would be required for our user:

• id : Int64 - auto-generated ID to uniquely identity each user
• first_name : String - first name of the user
• last_name : String - last name of the user
• email : String - email of the user, also used for login. Should be unique for each user
• password : String - the user's password
• created_at : Time - when the user was created
• updated_at : Time - when the user was updated (name/email/password change)
• deleted_at : Time - when the user was deleted

Translating this into a SQL statement:

CREATE TABLE "blog"."users"
(
    "id"         BIGSERIAL NOT NULL PRIMARY KEY,
    "first_name"  TEXT      NOT NULL,
    "last_name"  TEXT      NOT NULL,
    "email"      TEXT      NOT NULL,
    "password"   TEXT      NOT NULL,
    "created_at" TIMESTAMP NOT NULL DEFAULT NOW(),
    "updated_at" TIMESTAMP NOT NULL DEFAULT NOW(),
    "deleted_at" TIMESTAMP NULL
);

Now that our table is created, we can move on to create our first model. I will start by making a new directory to store our models; as well as creating our user.cr file.

$ mkdir ./src/models
$ touch ./src/models/user.cr

Using our list as reference I will create the user model.

@[ASRA::ExclusionPolicy(:all)]
class Blog::Models::User < Granite::Base
  include ASR::Serializable
  include Assert

  connection "my_blog"
  table "users"

  column id : Int64, primary: true
  column first_name : String
  column last_name : String
  column email : String
  column password : String
  column created_at : Time
  column updated_at : Time
  column deleted_at : Time?
end

A few things to point out:

• The connection macro defines which adapter this model should use to connect to the database. The value passed to the macro is the same as the name set when registering the DB adapter in blog.cr.
• I added a Models namespace just to add some organization and help separate the docs. Because of this be sure to add a include Models within the Blog module in blog.cr as well as a require "./models/*".
• We'll get back to the include ASR::Serializable and include Assert later.

Let's do a little recap. What have we done so far?

1. Registered our adapter to connect to our DB.
2. Required all the needed shards.
3. Created our User model and table.

Now that all of these beginning steps are done, we can now create our user_controller to hold our routes to create a user.

User Controller

Similarly as before, I will create a new directory to hold our controllers, as well as create our user_controller.cr file.

$ mkdir ./src/controllers
$ touch ./src/controllers/user_controller.cr

class Blog::Controllers::UserController < ART::Controller
end

I'm also namespacing the controllers, so be sure to include Controllers, as well as a require "./controllers/*" in your blog.cr file. The first endpoint I will create will be a POST /user endpoint in order to add users to our database. To do this, add the following code to the UserController class.

@[ART::Post("user")]
def new_user(user : Blog::Models::User) : Blog::Models::User
  user
end

Athena's route definitions are a bit different than what you may be used to. Athena uses Crystal's annotations. The top annotation defines a POST endpoint with the path /user and sets the route's action to the new_user method. However, Athena is not able to automatically provide complex types, such as our User object to our action; we must make use of a ParamConverter to accomplish this. A ParamConverter allows defining custom logic responsible for converting data within a request into another type for the action to use. In this example, convert the request body into an instance of our User model; lets go ahead and create that now.

# Define our converter, register it as a service, inheriting from the base interface struct.
@[ADI::Register]
struct Blog::Converters::RequestBody < ART::ParamConverterInterface
  # Define a customer configuration for this converter.
  # This allows us to provide a `model` field within the annotation
  # in order to define _what_ model should be used on deserialization.
  configuration model : Granite::Base.class

  # :inherit:
  def apply(request : HTTP::Request, configuration : Configuration) : Nil
    # Be sure to handle any possible exceptions here to return more helpful errors to the client.
    raise ART::Exceptions::BadRequest.new "Request body is empty" unless body = request.body.try &.gets_to_end

    # Deserialize the object, based on the type provided in the annotation
    obj = configuration.model.from_json body

    # Run the validations
    obj.validate!

    # Add the resolved object to the request's attributes
    request.attributes.set configuration.name, obj, configuration.model
  rescue ex : Assert::Exceptions::ValidationError
    # Raise a 422 error if the object failed its validations
    raise ART::Exceptions::UnprocessableEntity.new ex.to_s
  end
end

Athena uses a lot of interfaces in order to make types more DI friendly, easier to test, etc. The interface only requires a single method apply(request : HTTP::Request, configuration : Configuration) : Nil whose sole purpose is to apply the conversion logic to the provided request, based on the provided configuration. A converter is simply a struct that inherits from ART::ParamConverterInterface. We'll cover the @[ADI::Register] annotation a bit later.

Our RequestBody converter also makes use of Athena's error handling system. Athena provides a set of common HTTP exceptions inheriting from ART::Exceptions::HTTPException, children of this type are assumed to map to an HTTP error; custom children can also be added. Non HTTPExceptions return a 500 unless rescued as you would normally. By default the exceptions are JSON serialized, but can be customized if so desired.

Most commonly, param converters will want to store the converted values within the request's attributes. The attributes are held within an ART::ParameterBag instance. The ParameterBag is a container for storing key/value pairs; which can be used to store arbitrary data within the context of a request. By default, Athena will look in the request's attributes for a value with the same name as an action argument; this include path/query params, or any custom values stored in it.

Now that we have our converter defined we can go ahead to implement it on our new_user route. Simply apply the annotation, the first argument maps to the name of the action argument the converter should be applied against, while the converter named argument accepts the specific ParamConverter.class we want to use. Any extra configuration for this converter can also be defined. In this case we are specifying that we want to deserialize the request body into a User object.

Since the param converter supplies an actual User model object, we can just call .save in our action to save the given object, then just return the user object. We can also use the ART::View annotation to make our action a bit more REST friendly by having the action return a 201 Created status code instead of the standard 200 OK. Our new_user action now looks like:

@[ART::Post("user")]
@[ART::View(status: :created)]
@[ART::ParamConverter("user", converter: Blog::Converters::RequestBody, model: Blog::Models::User)]
def new_user(user : Blog::Models::User) : Blog::Models::User
  user.save
  user
end

At this point we are now able to create users via our POST /user endpoint. Lets give it a try.

Start the HTTP server.

$ crystal ./src/blog.cr

Lets register a user:

curl --request POST \
  --url http://localhost:3000/user \
  --header 'content-type: application/json' \
  --data '{
  "first_name": "foo",
  "last_name": "bar",
  "email": "[e-mail]",
  "password": "monkey123"
}'

Success! The user was persisted and now has an id and timestamps.

{
  "id": 1,
  "first_name": "foo",
  "last_name": "bar",
  "email": "[e-mail]",
  "password": "monkey123",
  "created_at": "2020-07-11T22:42:33Z",
  "updated_at": "2020-07-11T22:42:33Z"
}

However there are a few problems with the current implementation.

1. What would stop someone from setting their password/name/email as an empty string?

   1. Sure we could rely upon the front end for the validation, but that is easy to bypass.

2. We probably shouldn't be displaying the user's password in cleartext, let alone return it in the response.

3. What happens if someone were to POST twice with the same email? Since we are using the email as our user facing unique identifier, we should handle this.

Lets go back to our User model to address some of these issues. This is where ASR::Serializable and Assert comes into use.

NOTE: Assert will eventually be moved into the athena-framework organization as an independent component for validation.

We can update our model to look like:

@[ASRA::ExclusionPolicy(:all)]
class Blog::Models::User < Granite::Base
  include ASR::Serializable
  include Assert

  connection "my_blog"
  table "users"

  has_many articles : Article

  @[ASRA::Expose]
  @[ASRA::ReadOnly]
  column id : Int64, primary: true

  @[ASRA::Expose]
  @[Assert::NotBlank]
  column first_name : String

  @[ASRA::Expose]
  @[Assert::NotBlank]
  column last_name : String

  @[ASRA::Expose]
  @[Assert::NotBlank]
  @[Assert::Email(mode: :html5)]
  column email : String

  @[ASRA::IgnoreOnSerialize]
  @[Assert::Size(Range(Int32, Int32), range: 8..25, min_message: "Your password is too short", max_message: "Your password is too long")]
  column password : String

  @[ASRA::Expose]
  @[ASRA::ReadOnly]
  column created_at : Time

  @[ASRA::Expose]
  @[ASRA::ReadOnly]
  column updated_at : Time

  column deleted_at : Time?
end

Also update your new_user action to be:

@[ART::Post("user")]
@[ART::ParamConverter("user", converter: Blog::Converters::RequestBody, model: Blog::Models::User)]
def new_user(user : Blog::Models::User) : Blog::Models::User
  raise ART::Exceptions::Conflict.new "A user with this email already exists." if User.exists? email: user.email
  user.save
  user
end

With these changes we have addressed issues 1 and 3 that we identified earlier. We will solve issue 2 shortly after an explanation of what is going on.

Firstly we included ASR::Serializable and Assert in order to add enhanced serialization and assertion functionality. Next, we added an annotation to the class of our User model. @[ASRA::ExclusionPolicy(:all)]. This annotation alters the overall serialization strategy for the model. In this case, it will only serialize fields that are exposed via @[ASRA::Expose]. This is handy, especially for larger models, to make it easier to only serialize the expected fields, as well as prevent the serialization of other instance variables included via other modules for example.

I also added the @[ASRA::IgnoreOnSerialize] annotation to the password property. This tells Athena's serializer that the password is allowed to be deserialized, but should NOT be serialized.

Next, I have added annotations to expose the fields that we wish to be returned. I also added a @[ASRA::ReadOnly] to the id field and exposed timestamp fields, which prevents that property from being deserialized; since it's managed by the database.

I also added additional annotations to the fields we wish to validate. I am asserting that:

• The first_name field is not blank
• The last_name field is not blank
• The email is not blank AND is a valid email
• The password is between 8 and 25 characters long

Finally, I added a User.exists? email: user.email query in the UserController to check if a user exists with the given email, and throw a proper error message if one does.

Lets test it out!

curl --request POST \
  --url http://localhost:3000/user \
  --header 'content-type: application/json' \
  --data '{
  "first_name": "foo",
  "last_name": "",
  "email": "",
  "password": "monkey"
}'

produces the following response:

{
  "code": 422,
  "message": "Validation tests failed: 'last_name' should not be blank, 'email' is not a valid email address, 'email' should not be blank, Your password is too short"
}

Tada! Easy validation of your models. Also, trying to POST a user with an email that was used before now produces this error

{
  "code": 409,
  "message": "A user with this email already exists."
}

Issue 2 can be solved by adding a before_save callback on our model

@[ASRA::ExclusionPolicy(:all)]
class Blog::Models::User < Granite::Base
  ...

  before_save :hash_password

  def hash_password : Nil
    if p = @password
      @password = Crypto::Bcrypt::Password.create(p).to_s
    end
  end
end

This will execute and hash the password before the model is saved.

Auth Controller

At this point we now have a POST /user endpoint that would be paired with a front end form for user registration. But in order for the user to be "logged in" we need to do something to tell the front end that there is an active session. There are a multiple of ways to do this: setting a JWT token in a cookie, generating a session key and storing that in our user table, or returning a JWT token to the front end after receiving a correct username and password for the front end to store in some form of HTML5 storage. For the purposes of this I am going to go with the latter option, and return a JWT token for the front end to handle.

To start, I am going to create a new controller file under our ./src/controllers directory to hold our logic for signing in.

$ touch ./src/controllers/auth_controller.cr

I am also going to take this time to show off some additional features; namely working with the raw HTTP::Request object, and introduce ART::Response.

class Blog::Controllers::AuthController < ART::Controller
  # Type hinting an action argument to `HTTP::Request` will supply the current request object.
  @[ART::Post("login")]
  def login(request : HTTP::Request) : ART::Response
    # Raise an exception if there is no request body
    raise ART::Exceptions::BadRequest.new "Missing request body." unless body = request.body

    # Parse the request body into an HTTP::Params object
    form_data = HTTP::Params.parse body.gets_to_end

    # Handle missing form values
    handle_invalid_auth_credentials unless email = form_data["email"]?
    handle_invalid_auth_credentials unless password = form_data["password"]?

    # Find a user with the given ID
    user = Blog::Models::User.find_by email: email

    # Raise a 401 error if either a user isn't found or the password does not match
    handle_invalid_auth_credentials if !user || !(Crypto::Bcrypt::Password.new(user.password).verify password)

    # If an `ART::Response` is returned then it is used as is for the response,
    # otherwise, like the other endpoints, the response value is by default JSON serialized
    ART::Response.new({token: user.generate_jwt}.to_json, headers: HTTP::Headers{"content-type" => "application/json"})
  end

  private def handle_invalid_auth_credentials : Nil
    # Raise a 401 error if values are missing, or are invalid;
    # this also handles setting an appropiate `www-authenticate` header
    raise ART::Exceptions::Unauthorized.new "Invalid username and/or password.", "Basic realm=\"My Blog\""
  end
end

The raw request object can be obtained by type hinting an action argument as HTTP::Request, Athena will then know to provide the request object when executing the action. We then validate all the required fields are present, and a user was found with the given credentials; otherwise we return a 401 error for the front end to handle.

One thing to note is the return type in this action is ART::Response. At a high level, the implementation of Athena is simply attempting to convert an HTTP::Request into an ART::Response. If an action returns an ART::Response then the request is essentially finished and returned as is, (assuming no listeners alter it further, more on that later). Otherwise, like our other actions, if the return type is not an ART::Response, the resulting value goes through the view layer in order to have that value converted into an ART::Response. By default this is JSON serializing it, but it can be customized if so desired.

Next, we will need to implement the generate_jwt method on our user object, which will look like:

def generate_jwt : String
  JWT.encode({
    "user_id" => @id,
    "exp"     => (Time.utc + 1.week).to_unix,
    "iat"     => Time.utc.to_unix,
  },
    ENV["SECRET"],
    :hs512
  )
end

This method will generate a JWT with a body including:

• The id of the user
• An expiration date of now + 1 week
• The time the JWT was created

I generated a secure string and exported it as an env variable to act as the secret key to sign the token.

$ export SECRET=MY_SECURE_STRING

This is just a simple example, and not representative of how to best use JWT tokens. If this were for real you could include other claims or change the details to best fit your use cases.

After restarting the server and sending a request:

curl --request POST \
  --url http://localhost:3000/login \
  --header 'content-type: application/x-www-form-urlencoded' \
  --data 'email=fakeemail%40domain.com&password=monkey123'

You should get a JSON object back with your JWT token within it. Success! However, if you used invalid credentials you would get a 401 error back.

{
  "code": 401,
  "message": "Invalid username and/or password"
}

Since our imaginary front end will be storing this token in local storage on the browser, we don't really have a use for a /logout endpoint. However, if you wanted to make one you could have it issue a request before deleting the token from the front end. This way you tell your server that a given user logged out if you had other tasks/cleanup to do.

Article Model

At this point we are able to register new users, allow users to login, all the while validating and throwing helpful errors.

The next item on the agenda will be to create our Article model, table, and controller. I'll go a bit faster now as it'll be quite similar as before.

Next lets think about what columns would be required for an article:

• id : Int64 - auto-generated ID to uniquely identity each article
• user_id : Int64 - the user that authored the article
• title : String - title of the article
• body : String - the body
• created_at : Time - when the article was created
• updated_at : Time - when the article was updated
• deleted_at : Time - when the article was deleted

Translating this into a SQL statement:

CREATE TABLE "blog"."articles"
(
    "id"         BIGSERIAL NOT NULL PRIMARY KEY,
    "user_id"    BIGINT    NOT NULL REFERENCES "blog"."users",
    "title"      TEXT      NOT NULL,
    "body"       TEXT      NOT NULL,
    "created_at" TIMESTAMP NOT NULL DEFAULT NOW(),
    "updated_at" TIMESTAMP NOT NULL DEFAULT NOW(),
    "deleted_at" TIMESTAMP NULL
);

Now that our table is created, we can move on to create our second model. I will first create the article.cr file.

$ touch ./src/models/article.cr

@[ASRA::ExclusionPolicy(:all)]
class Blog::Models::Article < Granite::Base
  include ASR::Serializable
  include Assert

  connection my_blog
  table "articles"

  @[ASRA::Expose]
  @[ASRA::ReadOnly]
  belongs_to user : User

  @[ASRA::Expose]
  @[ASRA::ReadOnly]
  column id : Int64, primary: true

  @[ASRA::Expose]
  @[Assert::NotBlank]
  @[Assert::NotNil]
  column title : String

  @[ASRA::Expose]
  @[Assert::NotBlank]
  @[Assert::NotNil]
  column body : String

  @[ASRA::Expose]
  @[ASRA::ReadOnly]
  column updated_at : Time

  @[ASRA::Expose]
  @[ASRA::ReadOnly]
  column created_at : Time

  @[ASRA::ReadOnly]
  column deleted_at : Time?
end

This model is very similar to our User model with the main difference being the belongs_to user : User macro. This macro expands and creates the user_id field. It also creates a getter and setter to retrieve and set the related user object. In this case, the person who authored the article.

We'll also want to go back to the User model and add has_many articles : Article to it, below the table definition. This defines a method that would return an array of that user's articles. E.x. articles = user.articles.

Next up, the ArticleController.

Article Controller

$ touch ./src/controllers/article_controller.cr

class Blog::Controllers::ArticleController < ART::Controller
  @[ART::Post("article")]
  @[ART::View(status: :created)]
  @[ART::ParamConverter("article", converter: Blog::Converters::RequestBody, model: Blog::Models::Article)]
  def new_article(article : Blog::Models::Article) : Blog::Models::Article
    article.save
    article
  end
end

Again, this looks nearly the same as the new_user action in our UserController.

Making a request to create an article with a user_id of the id of your user, and the token retrieved earlier:

curl --request POST \
  --url http://localhost:3000/article \
  --header 'content-type: application/json' \
  --header 'authorization: Bearer TOKEN' \
  --data '{
    "user_id": 1,
    "title": "My Athena Blog",
    "body": "Athena makes developing JSON APIs easy!"
}'

Successfully creates the article:

{
  "user_id": 1,
  "id": 1,
  "title": "My Athena Blog",
  "body": "Athena makes developing JSON APIs easy!",
  "updated_at": "2020-07-11T22:57:56Z",
  "created_at": "2020-07-11T22:57:56Z"
}

Great! We can now create articles. However, do you see a problem with this implementation? There is no validation around the user_id, nor is there any authorization to prevent random people from creating articles for any user they want. Lets work on adding some authorization checks into our request flow, utilizing the generated JWT token we got a little while ago when we "logged in".

Authorization

One of the core points of JWT is that once verified, using our secret key and checking the claims in the body, it can be assured that it is a valid token and that we should process the request. Also, since this is a REST API, we'll need to enable CORS to allow our front end to actually make requests to it. We can accomplish the latter by enabling Athena's CORS listener.

We just need to simply define some configuration on how we want the listener to operate, see ART::Config::CORS for additional configuration information. Create a file in the root of your application named athena.yml with the following content:

---
routing:
  cors:
    allow_credentials: true
    allow_origin: 
      - https://api.myblog.com
    allow_methods:
      - GET
      - POST
      - PUT
      - DELETE

Athena uses Athena::EventDispatcher to handle tapping into the request/response life-cycle, as opposed to the more standard HTTP::Handler approach.

When processing a request, Athena emits various events that can be listened on to handle the request early, like the CORS listener, or for adding additional information to the response, like headers/cookies etc. A good example of this would be to tap into when an unhandled exception occurs for logging purposes.

For our goal of authenticating a user, we will create a listener on the Request event. This will be used to validate there is a token present and that it is valid. Lets get started.

First, lets make a new directory to store our handler.

$ mkdir ./src/listeners
$ touch ./src/listeners/security_listener.cr

struct Blog::Listeners::SecurityListener
  # Define the interface to implement the required methods
  include AED::EventListenerInterface

  # Specify that we want to listen on the `Request` event.
  # The value of the has represents this listener's priority;
  # the higher the value the sooner it gets executed.
  def self.subscribed_events : AED::SubscribedEvents
    AED::SubscribedEvents{
      ART::Events::Request => 10,
    }
  end

  # Define a `#call` method scoped to the `Request` event.
  def call(event : ART::Events::Request, _dispatcher : AED::EventDispatcherInterface) : Nil
    # Allow POST user and POST login through since they are public
    # In the future Athena will most likely have a more structured way to handle auth
    if event.request.method == "POST" && {"/user", "/login"}.includes? event.request.path
      return
    end

    # Return a 401 error if the token is missing or malformed
    raise ART::Exceptions::Unauthorized.new "Missing bearer token", "Bearer realm=\"My Blog\"" unless (auth_header = event.request.headers.get?("Authorization").try &.first) && auth_header.starts_with? "Bearer "

    # Get the JWT token from the Bearer header
    token = auth_header.lchop "Bearer "

    begin
      # Validate the token
      body = JWT.decode token, ENV["SECRET"], :hs512
    rescue decode_error : JWT::DecodeError
      # Throw a 401 error if the JWT token is invalid
      raise ART::Exceptions::Unauthorized.new "Invalid token", "Bearer realm=\"My Blog\""
    end
  end
end

Now that our listener is defined, we're faced with some new problems.

1. How do we tell Athena to use it?
2. How can we make the rest of our application aware of the currently authenticated user?

Both of these problems are solved via another feature of Athena, dependency injection (DI). Athena uses Athena::DependencyInjection to make sharing useful objects easy. While I'm going to cover the main points of DI in this article, see Dependency Injection in Crystal, in addition to the API docs within the shard, for a more detailed example of it in action.

A service container contains instances of various useful object, aka services. These services can then be supplied to other services without having to manually instantiate everything. It also allows for types to be tested more easily since they can depend on abstractions (interfaces) versus concrete types. In our case it'll allow us to define a service that will store the currently authenticated user in order to have access to it in the rest of the application.

First let's define a type to store the user, aka UserStorage. We'll make a new directory to store our services that don't fit better anywhere else.

$ mkdir ./src/services
$ touch ./src/services/user_storeage.cr

# The ADI::Register annotation tells the DI component how this service should be registered
@[ADI::Register]
class Blog::UserStorage
  # Use a ! property since they'll always be a user defined in our use case.
  #
  # It also provides a `user?` getter in cases where it might not be.
  property! user : Blog::Models::User
end

Since we defined this as a class, it makes it so the same instance is injected into each type, i.e. the user initially set will remain set until the request is finished. A struct on the other hand would cause a copy of the service to be injected.

Be sure to require our new directory in src/blog.cr.

Now lets update our security listener, I omitted the lines that didn't change.

# Define and register a listener to handle authenticating requests.
@[ADI::Register]
struct Blog::Listeners::SecurityListener
  # Define the interface to implement the required methods
  include AED::EventListenerInterface

  # Define our initializer for DI to inject the user storage.
  def initialize(@user_storage : UserStorage); end

  # Define a `#call` method scoped to the `Request` event.
  def call(event : ART::Events::Request, _dispatcher : AED::EventDispatcherInterface) : Nil
    ...

    # Set the user in user storage
    @user_storage.user = Blog::Models::User.find! body[0]["user_id"]
  end
end

By simply annotating the type with @[ADI::Register], Athena handles "wiring" everything up for us. Our required UserStorage dependency is automatically resolved and injected based on type restriction. The listener is also registered automatically since it implements AED::EventListenerInterface, via ADI.auto_configure.

Now that we are authenticating the requests, we can move onto adding the ability to read/update/delete our articles.

Our ArticleController now looks like:

# The `ART::Prefix` annotation will add the given prefix to each route in the controller.
# We also register the controller itself as a service in order to allow injecting our `UserStorage` object.
# NOTE: The controller service must be declared as public.  In the future this will happen behind the scenes
# but for now it cannot be done automatically.
@[ART::Prefix("article")]
@[ADI::Register(public: true)]
class Blog::Controllers::ArticleController < ART::Controller
  # Define our initializer for DI
  def initialize(@user_storage : Blog::UserStorage); end

  @[ART::Post(path: "")]
  @[ART::View(status: :created)]
  @[ART::ParamConverter("article", converter: Blog::Converters::RequestBody, model: Blog::Models::Article)]
  def new_article(article : Blog::Models::Article) : Blog::Models::Article
    # Set the owner of the article to the currently authenticated user
    article.user = @user_storage.user
    article.save
    article
  end

  @[ART::Get(path: "")]
  def get_articles : Array(Blog::Models::Article)
    # We are also using the user in UserStorage as an additional conditional in our query when fetching articles
    # this allows us to only returns articles that belong to the current user.
    Blog::Models::Article.where(:deleted_at, :neq, nil).where(:user_id, :eq, @user_storage.user.id).select
  end

  @[ART::Put(path: "")]
  @[ART::ParamConverter("article", converter: Blog::Converters::RequestBody, model: Blog::Models::Article)]
  def update_article(article : Blog::Models::Article) : Blog::Models::Article
    article.save
    article
  end

  @[ART::Get("/:id")]
  @[ART::ParamConverter("article", converter: Blog::Converters::DB, model: Blog::Models::Article)]
  def get_article(article : Blog::Models::Article) : Blog::Models::Article
    article
  end

  @[ART::Delete("/:id")]
  @[ART::ParamConverter("article", converter: Blog::Converters::DB, model: Blog::Models::Article)]
  def delete_article(article : Blog::Models::Article) : Nil
    article.deleted_at = Time.utc
    article.save
  end
end

These additional methods will allow for:

• Listing all the current user's articles
• Updating an article
• Getting a specific article
• Deleting a specific article

The latter two are making use of a new converter; DB. This will do a DB query to find a record of the provided type with the provided id, otherwise returns a 404 error. The code for that is as follows:

@[ADI::Register]
struct Blog::Converters::DB < ART::ParamConverterInterface
  # Define a customer configuration for this converter.
  # This allows us to provide a `model` field within the annotation
  # in order to define _what_ model should be queried for.
  configuration model : Granite::Base.class

  # :inherit:
  #
  # Be sure to handle any possible exceptions here to return more helpful errors to the client.
  def apply(request : HTTP::Request, configuration : Configuration) : Nil
    # Grab the `id` path parameter from the request's attributes
    primary_key = request.attributes.get "id", Int32

    # Raise a 404 if a record with the provided ID does not exist
    raise ART::Exceptions::NotFound.new "An item with the provided ID could not be found" unless model = configuration.model.find primary_key

    # Set the resolved model within the request's attributes
    # with a key matching the name of the argument within the converter annotation
    request.attributes.set configuration.name, model, configuration.model
  end
end

Updating DB models can be a bit tricky in some cases due to how Crystal handles deserialization. In other frameworks it would be required to include ALL the properties of the model in the PUT body, even those managed by the database that should not be editable, such as the id or timestamps. Athena's serializer includes the concept of Object Constructors; which determine how a new object is constructed during deserialization. In our case, we could define a custom constructor that would source the object from the database, making it so we don't need to include the timestamps, or other non-editable properties within our PUT request.

Within request_body_converter.cr add the following code:

# Define a custom `ASR::ObjectConstructorInterface` to allow sourcing the model from the database
# as part of `PUT` requests, and if the type is a `Granite::Base`.
#
# Alias our service to `ASR::ObjectConstructorInterface` so ours gets injected instead.
@[ADI::Register(alias: ASR::ObjectConstructorInterface)]
class DBObjectConstructor
  include Athena::Serializer::ObjectConstructorInterface

  # Inject `ART::RequestStore` in order to have access to the current request.
  # Also inject `ASR::InstantiateObjectConstructor` to act as our fallback constructor.
  def initialize(@request_store : ART::RequestStore, @fallback_constructor : ASR::InstantiateObjectConstructor); end

  # :inherit:
  def construct(navigator : ASR::Navigators::DeserializationNavigatorInterface, properties : Array(ASR::PropertyMetadataBase), data : ASR::Any, type)
    # Fallback on the default object constructor if the type is not a `Granite` model.
    unless type <= Granite::Base
      return @fallback_constructor.construct navigator, properties, data, type
    end

    # Fallback on the default object constructor if the current request is not a `PUT`.
    unless @request_store.request.method == "PUT"
      return @fallback_constructor.construct navigator, properties, data, type
    end

    # Lookup the object from the database; assume the object has an `id` property.
    object = type.find data["id"].as_i

    # Return a `404` error if no record exists with the given ID.
    raise ART::Exceptions::NotFound.new "An item with the provided ID could not be found." unless object

    # Apply the updated properties to the retrieved record
    object.apply navigator, properties, data

    # Return the object
    object
  end
end

# Make the compiler happy when we want to allow any Granite model to be deserializable.
class Granite::Base
  include ASR::Model
end

This type allows us to source the original object from the database, then apply the updated values to it as opposed to creating a whole new object from the request body. The DB logic is only applied to Granite::Base types on PUT requests, everything else uses the default behavior of creating a new object with based on the data within the request body.

We can then update our RequestBody converter to look like:

# Define our converter, register it as a service, inheriting from the base interface struct.
@[ADI::Register]
struct Blog::Converters::RequestBody < ART::ParamConverterInterface
  # Define a custom configuration for this converter.
  # This allows us to provide a `model` field within the annotation
  # in order to define _what_ model should be used on deserialization.
  configuration model : Granite::Base.class

  # Inject the Serializer instance into our converter.
  def initialize(@serializer : ASR::SerializerInterface); end

  # :inherit:
  def apply(request : HTTP::Request, configuration : Configuration) : Nil
    # Be sure to handle any possible exceptions here to return more helpful errors to the client.
    raise ART::Exceptions::BadRequest.new "Request body is empty." unless body = request.body.try &.gets_to_end

    # Deserialize the object, based on the type provided in the annotation.
    object = @serializer.deserialize configuration.model, body, :json

    # Run the validations.
    object.validate!

    # Add the resolved object to the request's attributes.
    request.attributes.set configuration.name, object, configuration.model
  rescue ex : Assert::Exceptions::ValidationError
    # Return a `422` error if the object failed its validations.
    raise ART::Exceptions::UnprocessableEntity.new ex.to_s
  end
end

From here you would want to update ArticleController#update_article with some ACL logic, such as:

  @[ART::Put(path: "")]
  @[ART::ParamConverter("article", converter: Blog::Converters::RequestBody, model: Blog::Models::Article)]
  def update_article(article : Blog::Models::Article) : Blog::Models::Article
    # Ensure that a user cannot edit someone else's article
    raise ART::Exceptions::Forbidden.new "Only the author of the article can edit it." if article.user_id != @user_storage.user.id
    article.save
    article
  end

And we're done! I hope this was a good introduction to Athena and its features. If there is anything in specific you would like to see regarding Granite/Athena, or if I missed something, feel free to leave a comment or join the Athena Gitter channel.

The full code for this tutorial is available on GitHub.

Comments

[phangs]

I tried following this tutorial to learn about crystal and api. I am newbie programmer btw.

Following the steps manually and cloning the github source yield the same error:

In src/blog.cr:2:1

2 | Granite::Adapters << Granite::Adapter::Pg.new({name: "my_blog", url: "postgres://dbadmin:password@localhost:5432/blog?currentSchema=blog"})
^{----------------}
Error: undefined constant Granite::Adapters

Did you mean 'Granite::Adapter'?

can you please help what I am doing wrong?

[Blacksmoke16]

This is a result of some breaking changes that happened in the new Granite version. See github.com/amberframework/granite/...

I'll update the guide to reflect those changes.

[phangs]

Thank you! Looking forward to the update

[Blacksmoke16]

Should be good to go now, let me know if you run into any trouble.

[phangs]

tried the source again from the repo, new error:

Athena::Routing::Converters::Athena::Routing::Converters::RequestBody(Blog::Models::Article, Nil).new.convert val
^{--------------------------------------------------------------------}
Error: undefined constant Athena::Routing::Converters::Athena::Routing::Converters::RequestBody

I'm still figuring out the problem just would like to let you know.

[Blacksmoke16]

Make sure you do a shards update I updated some dependencies and pinned the versions so it'll always use the correct version.

[phangs]

tried again with the shards update. I still couldn't figure out. sorry, I am new to programming, I am a business analyst, and was thinking I could grow with crystal as my realy programming language that's why I am trying to learn it.

[phangs]

@routes.add "/POST/user", RouteAction(

846 | # Map Nil return type to Noop to avoid github.com/crystal-lang/crystal/is...
847 | Proc(HTTP::Server::Context, Hash(String, String?), Blog::Models::User), Athena::Routing::Renderers::JSONRenderer, Blog::Controllers::UserController)
848 | .new(
849 | __temp_647,
850 | RouteDefinition.new("/POST/user", nil),
851 | Callbacks.new([] of CallbackBase, [] of CallbackBase),
852 | "new_user",
853 | ["default"],
854 | [Athena::Routing::Parameters::BodyParameter(Blog::Models::User).new("body")] of Athena::Routing::Parameters::Param
855 | )

[Blacksmoke16]

what's the actual error you're getting? Should be towards the very bottom.

I ran through the tutorial on crystal 0.31.1 and everything was fine, so also make sure that's up to date.

[phangs]

`wilbert@wilbert-UX360CAK:~/Documents/Development/crystal/athena-blog-tutorial$ crystal ./src/blog.cr
Showing last frame. Use --error-trace for full trace.

There was a problem expanding macro 'macro_140521444230352'

Code in lib/athena/src/routing/handlers/route_handler.cr:19:7

19 | {% for klass in Athena::Routing::Controller.all_subclasses %}
^
Called macro defined in lib/athena/src/routing/handlers/route_handler.cr:19:7

19 | {% for klass in Athena::Routing::Controller.all_subclasses %}

Which expanded to:

140 | arr << if val = vals[key]?
141 |

142 | Athena::Routing::Converters::Athena::Routing::Converters::RequestBody(Blog::Models::Article, Nil).new.convert val
^{--------------------------------------------------------------------}
Error: undefined constant Athena::Routing::Converters::Athena::Routing::Converters::RequestBody
wilbert@wilbert-UX360CAK:~/Documents/Development/crystal/athena-blog-tutorial$`

[Blacksmoke16]

Ahhh I figured it out. Apparently shards update doesn't actually update the directory in ./lib, thus my version locally was still using the older Athena version. I'll push a fix right now.

[phangs]

that's great! thank you very much. I will check in a bit and clone the project again.

I really appreciate your help

[phangs]

Code is now running, just noticed:

1. After cloning, user must create the logs/development.log directory and file. The code will look for it and will not compile it does not exist.

2. Was not yet able to find the problem when send post request to localhost:8888/user, error in logs is:
   [2019-11-25T05:37:51.194843000Z] main.CRITICAL: Unhandled exception: relation "users" does not exist in Blog::Controllers::UserController at src/controllers/user_controller.cr:6:107 {"cause":null,"cause_class":"Nil"}

I will try later to figure this out

[Blacksmoke16]

Thanks, I pushed a fix for #1. The other error would be because Granite can't find a table called users in the database you're connected to. Be sure you ran the few SQL scripts I've included if you're using PG. Otherwise, be sure you create tables in your DB of choice.