DEV Community: Tore Pettersen

Rust Web Development Tutorial: Macros

Tore Pettersen — Sun, 23 Feb 2020 09:39:44 +0000

Originaly posted on cloudmaker.dev

In this tutorial, we are going to create a couple of macros to avoid repeating ourselves all the time and to make our code a bit cleaner. The code we are going to clean up is from the tutorial on sorting and filtering, so I highly recommend reading that first or cloning the code from github.

Define our first macro for sorting

The macros we will create are for helping us with database functions like sorting and filtering. We will start by creating a new file in the db folder and create our first macro to help us with sorting.

// src/db/macros.rs
#[macro_export]
macro_rules! sort_by {
   ($query:expr, $sort_by:expr, $(($param:expr, $column:expr)),*) => {
       {
           if let Some(sort_by) = $sort_by {
               $query = match sort_by.as_ref() {
                   $(
                       $param => $query.order($column.asc()),
                       concat!($param, ".asc") => $query.order($column.asc()),
                       concat!($param, ".desc") => $query.order($column.desc()),
                   )*
                   _ => $query,
               }
           }
           $query
       }
   };
}

This might be a lot of new syntax, but I will not go too deep into the details since that is already explained quite well in the official docs. But in short, we are defining a macro that expects a query and the parameter that we will use to sort by. Also, we need to define which values we allow the parameter to be and which column the value is referring to. Then we just use this information to rebuild a more generic query.

Don’t forget to add the macro module to the db/mod.rs file, before we continue:

// src/db/mod.rs
mod connection;
mod paginate;
mod macros;

pub use connection::*;
pub use paginate::*;

Now let’s replace our existing implementation for sorting and instead use our new macro:

// src/user/model.rs
use crate::sort_by;
// ..

impl User {
   pub fn find_all(params: Params) -> Result<(Vec<Self>, i64), ApiError> {
       let conn = db::connection()?;

       let mut query = user::table.into_boxed();

       if let Some(email) = params.email {
           query = query.filter(user::email.like(email));
       }
       if let Some(created_at_gte) = params.created_at_gte {
           query = query.filter(user::created_at.ge(created_at_gte));
       }
       if let Some(created_at_lte) = params.created_at_lte {
           query = query.filter(user::created_at.le(created_at_lte));
       }
       if let Some(updated_at_gte) = params.updated_at_gte {
           query = query.filter(user::updated_at.ge(updated_at_gte));
       }
       if let Some(updated_at_lte) = params.updated_at_lte {
           query = query.filter(user::updated_at.le(updated_at_lte));
       }

       query = sort_by!(query, params.sort_by,
           ("id", user::id),
           ("email", user::email),
           ("created_at", user::created_at),
           ("updated_at", user::updated_at)
       );

       let (users, total_pages) = query
           .load_with_pagination(&conn, params.page, params.page_size)?;

       Ok((users, total_pages))
   }
   // ..

For the sorting we now went down from 17 lines to 6. That is quite an improvement and makes it a bit easier to read. The more values we allow to sort by, the more lines we will save since we now only have to add one new line for each extra value instead of tree without the macro.
Also notice that the macro is being placed at the root of our crate instead of in the database module when we are adding it to the namespace.

Macro for filtering

The macro for filtering is going to be a bit more complex, since we need to have the option for selecting if we want to use gt, le or like to compare the values. But we can do that as well with a bit of creativity.

// src/db/macros.rs
#[macro_export]
macro_rules! filter {
   ($query:expr, $(($column:expr, @$expression_method:ident, $param:expr)),*) => {
       {
           $(
               if let Some(item) = $param {
                   let filter = filter!($column, @$expression_method, item);
                   $query = $query.filter(filter);
               }
           )*
           $query
       }
   };
   ($column:expr, @like, $item:expr) => { $column.like($item) };
   ($column:expr, @ge, $item:expr) => { $column.ge($item) };
   ($column:expr, @le, $item:expr) => { $column.le($item) };
}

As with our last macro we also have to pass the query into the macro for filtering. For each filter we want, we also need to pass the column, which expression method we are going to use and which parameter we want to compare with.
You probably notice that this new macro has several arms. It works a bit similar to the match statement, just that for the macros we are comparing complete statements instead of a single variable.

Also we have created our own keywords @like, @ge and @le, to differentiate the different expression methods. The reason we are using @ as a prefix is that it is not used in prefix position, meaning it will not conflict with anything.

Now let’s use our new macro.

use crate::{filter, sort_by};
// ..
impl User {
   pub fn find_all(params: Params) -> Result<(Vec<Self>, i64), ApiError> {
       let conn = db::connection()?;

       let mut query = user::table.into_boxed();

       query = filter!(query,
           (user::email, @like, params.email),
           (user::created_at, @ge, params.created_at_gte),
           (user::created_at, @le, params.created_at_lte),
           (user::updated_at, @ge, params.updated_at_gte),
           (user::updated_at, @le, params.updated_at_lte)
       );

       query = sort_by!(query, params.sort_by,
           ("id", user::id),
           ("email", user::email),
           ("created_at", user::created_at),
           ("updated_at", user::updated_at)
       );

       let (users, total_pages) = query
           .load_with_pagination(&conn, params.page, params.page_size)?;

       Ok((users, total_pages))
   }
   // ..

In this case we were able to reduce the filtering from 15 to 7 lines. But I feel that the even bigger win here is readability. Our custom keywords make it really easy to see which expression is used for each column and to which param it is compared with.

So hopefully now everything should work as before, but improved readability is always a plus.

And as always, you will be able to find the complete code on github.

Rust Web Development Tutorial: Pagination

Tore Pettersen — Sun, 16 Feb 2020 08:11:35 +0000

Originaly posted on cloudmaker.dev

In this tutorial, we are going to build pagination for our API results. To solve this problem we need to hook into Diesel’s query builder, so I will also cover how we can do that. This tutorial builds on my tutorial on building a REST API, so I highly recommend reading that first or cloning the code from github.

Extending the query builder

Since we are going to extend our database API, it would make sense to make a separate folder for that. So let’s rename and move the src/db.rs file into src/db/connection.rs. We also need to remember to create a mod file to continue to expose the methods outside of the module.

mod connection;
mod paginate;

pub use connection::*;
pub use paginate::*;

The pagination module is of course for our pagination and we will continue with that. Since diesel doesn’t support pagination out of the box, we have to extend the query builder ourselves.

For simplicity, we are going to use offset pagination, although that is not the most efficient on bigger data sets. But you should hopefully be able to use what you learn in this tutorial extend Diesel's query builder with queries for pagination that fits your use case. The query we are going to execute should be able to limit the number of entries that we get and to count the total entries. And we can do that with a query that looks like this:

SELECT *, COUNT(*) OVER () FROM (subselect t) LIMIT $1 OFFSET $2

To extend the query builder with this query, we need to create a struct that implements the QueryFragment trait. A struct that implements QueryFragment also needs to implement QueryId, which we can implement with the derive attribute.

The struct represents an executable query, so we will also implement RunQueryDsl which will add functions like execute and load. The query also has a return type, which we can state by implementing the Query trait.

use diesel::prelude::*;
use diesel::pg::Pg;
use diesel::query_builder::*;
use diesel::sql_types::BigInt;

const DEFAULT_PAGE_SIZE: i64 = 10;

#[derive(QueryId)]
pub struct Paginated<T> {
    query: T,
    page: i64,
    page_size: i64,
}

pub trait Paginate: Sized {
    fn paginate(self, page: i64) -> Paginated<Self>;
}

impl<T> Paginate for T {
    fn paginate(self, page: i64) -> Paginated<Self> {
        Paginated {
            query: self,
            page_size: DEFAULT_PAGE_SIZE,
            page,
        }
    }
}

impl<T> QueryFragment<Pg> for Paginated<T>
where
    T: QueryFragment<Pg>,
{
    fn walk_ast(&self, mut out: AstPass<Pg>) -> QueryResult<()> {
        out.push_sql("SELECT *, COUNT(*) OVER () FROM (");
        self.query.walk_ast(out.reborrow())?;
        out.push_sql(") t LIMIT ");
        out.push_bind_param::<BigInt, _>(&self.page_size)?;
        out.push_sql(" OFFSET ");
        let offset = (self.page - 1) * self.page_size;
        out.push_bind_param::<BigInt, _>(&offset)?;
        Ok(())
    }
}

impl<T: Query> Query for Paginated<T> {
    type SqlType = (T::SqlType, BigInt);
}

impl<T> RunQueryDsl<PgConnection> for Paginated<T> {}

Now we can use the paginate function on queries and load them into a Vec<(T, i64)>. So let’s try that on in the user API.

// src/user/model.rs
use crate::db::Paginate;
//..

#[derive(Debug, Deserialize)]
pub struct Params {
    pub page: Option<i64>,
    pub page_size: Option<i64>,
    // ..
}

impl User {
    pub fn find_all(params: Params) -> Result<(Vec<Self>, i64), ApiError> {
        let conn = db::connection()?;
        let mut query = user::table.into_boxed();
        // ..

        let (users, total_pages) = match params.page {
            Some(page) => {
                let res = query.paginate(page).load::<(User, i64)>(&conn)?;

                let total = res.get(0).map(|x| x.1).unwrap_or(0);
                let users = res.into_iter().map(|x| x.0).collect();
                let total_pages = (total as f64 / 10 as f64).ceil() as i64;

                (users, total_pages)
            },
            None => (query.load(&conn)?, 1),
        };

        Ok((users, total_pages))
    }
    // ..

Now we just have to pass the total pages back to the route before we can give it a try.

// src/user/routes.rs
// ..
#[get("/users")]
async fn find_all(filters: web::Query<Params>) -> Result<HttpResponse, ApiError> {
    let (users, total_pages) = User::find_all(filters.into_inner())?;
    Ok(HttpResponse::Ok().json(json!({"users": users, "total_pages": total_pages})))
}
// ..

Now we should be able to test our endpoint with the page parameter, but you might notice that we are still not able to change the page size. Also wouldn't it be nicer if we didn't had to write all this boilerplate code each time we add pagination. We can add a another trait and few functions to handle that for us.

// src/db/paginate.rs
use diesel::query_dsl::methods::LoadQuery;
use diesel::sql_types::HasSqlType;
// ..
impl<T> Paginated<T> {
    pub fn page_size(self, page_size: i64) -> Self {
        Paginated { page_size, ..self }
    }

    pub fn load_and_count_pages<U>(self, conn: &PgConnection) -> QueryResult<(Vec<U>, i64)>
    where
        Self: LoadQuery<PgConnection, (U, i64)>,
    {
        let page_size = self.page_size;
        let results = self.load::<(U, i64)>(conn)?;
        let total = results.get(0).map(|x| x.1).unwrap_or(0);
        let records = results.into_iter().map(|x| x.0).collect();
        let total_pages = (total as f64 / page_size as f64).ceil() as i64;
        Ok((records, total_pages))
    }
}

pub trait LoadPaginated<U>: Query + QueryId + QueryFragment<Pg> + LoadQuery<PgConnection, U> {
    fn load_with_pagination(self, conn: &PgConnection, page: Option<i64>, page_size: Option<i64>) -> QueryResult<(Vec<U>, i64)>;
}

impl<T, U> LoadPaginated<U> for T
where
    Self: Query + QueryId + QueryFragment<Pg> + LoadQuery<PgConnection, U>,
    U: Queryable<Self::SqlType, Pg>,
    Pg: HasSqlType<Self::SqlType>,
{
    fn load_with_pagination(self, conn: &PgConnection, page: Option<i64>, page_size: Option<i64>) -> QueryResult<(Vec<U>, i64)> {
        let (records, total_pages) = match page {
            Some(page) => {
                let mut query = self.paginate(page);
                if let Some(page_size) = page_size {
                    query = query.page_size(page_size);
                }

                query.load_and_count_pages::<U>(conn)?
            },
            None => (self.load::<U>(conn)?, 1),
        };

        Ok((records, total_pages))
    }
}

Now it should be a bit easier to add pagination with our LoadPaginated trait, which also allow us to add the parameter for page size.

// src/user/model.rs
use crate::db::LoadPaginated;
// ..
impl User {
    pub fn find_all(params: Params) -> Result<(Vec<Self>, i64), ApiError> {
        let conn = db::connection()?;
        let mut query = user::table.into_boxed();
        // ..

        let (users, total_pages) = query
            .load_with_pagination(&conn, params.page, params.page_size)?;

        Ok((users, total_pages))
    }
    // ..

Now we should also be able to use our API with the page size parameter as well. In case you need it, you can find the complete code on github.

Rust Web Development Tutorial: Sorting and Filtering

Tore Pettersen — Sun, 09 Feb 2020 09:19:16 +0000

Originaly posted on cloudmaker.dev

In this tutorial, we are going to allow our API results to be filtered and sorted. To help us solve this we are going to use Diesel’s query builder to build a conditional query. This tutorial builds on my tutorial on building a REST API, so I highly recommend reading that first or cloning the code from github.

We are going to start by creating a struct for the parameters that we allow to filter and sort by. Then we pass the filters to our API for finding users before we build up our conditional query.

// src/user/model.rs
//..
#[derive(Debug, Deserialize)]
pub struct Params {
    pub email: Option<String>,
    pub sort_by: Option<String>,
    #[serde(rename = "created_at[gte]")]
    pub created_at_gte: Option<NaiveDateTime>,
    #[serde(rename = "created_at[lte]")]
    pub created_at_lte: Option<NaiveDateTime>,
    #[serde(rename = "updated_at[gte]")]
    pub updated_at_gte: Option<NaiveDateTime>,
    #[serde(rename = "updated_at[lte]")]
    pub updated_at_lte: Option<NaiveDateTime>,
}

impl User {
    pub fn find_all(params: Params) -> Result<Vec<Self>, ApiError> {
        let conn = db::connection()?;

        let mut query = user::table.into_boxed();

        if let Some(email) = params.email {
            query = query.filter(user::email.like(email));
        }
        if let Some(created_at_gte) = params.created_at_gte {
            query = query.filter(user::created_at.ge(created_at_gte));
        }
        if let Some(created_at_lte) = params.created_at_lte {
            query = query.filter(user::created_at.le(created_at_lte));
        }
        if let Some(updated_at_gte) = params.updated_at_gte {
            query = query.filter(user::updated_at.ge(updated_at_gte));
        }
        if let Some(updated_at_lte) = params.updated_at_lte {
            query = query.filter(user::updated_at.le(updated_at_lte));
        }
        if let Some(sort_by) = params.sort_by {
            query = match sort_by.as_ref() {
                "id" => query.order(user::id.asc()),
                "id.asc" => query.order(user::id.asc()),
                "id.desc" => query.order(user::id.desc()),
                "email" => query.order(user::email.asc()),
                "email.asc" => query.order(user::email.asc()),
                "email.desc" => query.order(user::email.desc()),
                "created_at" => query.order(user::created_at.asc()),
                "created_at.asc" => query.order(user::created_at.asc()),
                "created_at.desc" => query.order(user::created_at.desc()),
                "updated_at" => query.order(user::updated_at.asc()),
                "updated_at.asc" => query.order(user::updated_at.asc()),
                "updated_at.desc" => query.order(user::updated_at.desc()),
                _ => query,
            };
        }

        let users = query
            .load::<User>(&conn)?;

        Ok(users)
    }
    //..
}

Notice that we are using .into_boxed() when we create the initial query. This is used to box the query into a single type so that the compiler knows how to deal with it when we build our conditional query.

You probably also notice that we have to be quite explicit about the behavior of our API. For every single parameter, we have to define the expected behavior and we also have to define every possible parameter for sorting. The good thing about that is that we know what to expect from our API and we will not end up with any surprising behavior.

If you are a bit like me you would probably also think that this kind of code is a bit messy. I feel that I have to repeat myself a bit too much by writing this kind of behavior over and over again when I write several APIs. To solve that problem we can use macros, but I will be covering that in a upcoming tutorial.
Next, we need to grab the parameters from the request and pass them to our API.

// src/user/routes.rs
//..
use crate::user::{User, UserMessage, Params};

#[get("/users")]
async fn find_all(params: web::Query<Params>) -> Result<HttpResponse, ApiError> {
    let users = User::find_all(params.into_inner())?;
    Ok(HttpResponse::Ok().json(users))
}
//..

Now we can give our API a try. Here are a few examples requests you can try and you will also be able to combine the parameters.

$ curl 'http://localhost:5000/users?email=john%'
$ curl 'http://localhost:5000/users?created_at[lte]=2019-12-11T00:00:00'
$ curl 'http://localhost:5000/users?sort_by=created_at.desc'

The complete source code of this tutorial is available on github in case you need it.

Rust Web Development Tutorial: Email Confirmations

Tore Pettersen — Sun, 02 Feb 2020 11:59:30 +0000

Originaly posted on cloudmaker.dev

In this tutorial, we are going to verify our API users with an email confirmation. This tutorial builds on the two previous tutorials on creating a REST API and authentication.

For this tutorial we are going to use Sendinblue and I would suggest activating the account right away since transactional email needs to be manually activated by support. Don’t worry though, when you have signed up, you just have to send a short email and let them know that you want transactional email activated. I only needed to wait two and a half hours, which I find quite fast for a service I don’t even pay for.

Creating a model for the email verification token

We are going to start to create a new migration for our email verification token. And we will of course do that with diesel-cli.

diesel migration generate email_verification_token

In the two generated files we add our migration scripts.

-- up.sql
CREATE TABLE email_verification_token (
    id BYTEA PRIMARY KEY,
    email TEXT UNIQUE NOT NULL,
    expires_at TIMESTAMP NOT NULL,
    created_at TIMESTAMP NOT NULL DEFAULT current_timestamp
);

-- down.sql
DROP TABLE email_verification_token;

Now that we have created the migration, we can execute it with the diesel migration run command.

Next we are going to create a model so that we have a representation of the email token in Rust. We are also going to implement methods for finding, creating and deleting the tokens.

// src/email_verification_token/model.rs
use crate::api_error::ApiError;
use crate::db;
use crate::schema::email_verification_token;
use chrono::{NaiveDateTime, Utc, Duration};
use diesel::prelude::*;
use rand::Rng;
use serde::{Deserialize, Serialize};

#[derive(Deserialize, Clone)]
pub struct EmailVerificationTokenMessage {
    pub id: Option<String>,
    pub email: String,
}

#[derive(Deserialize, Serialize, Queryable, Insertable)]
#[table_name = "email_verification_token"]
pub struct EmailVerificationToken {
    pub id: Vec<u8>,
    pub email: String,
    pub expires_at: NaiveDateTime,
    pub created_at: NaiveDateTime,
}

impl EmailVerificationToken {
    pub fn find(id: &Vec<u8>) -> Result<Self, ApiError> {
        let conn = db::connection()?;

        let token = email_verification_token::table
            .filter(email_verification_token::id.eq(id))
            .first(&conn)?;

        Ok(token)
    }

    pub fn create(body: EmailVerificationTokenMessage) -> Result<Self, ApiError> {
        let conn = db::connection()?;

        let id = rand::thread_rng().gen::<[u8; 32]>().to_vec();
        let email = body.email;
        let created_at = Utc::now().naive_utc();
        let expires_at = created_at + Duration::hours(12);
        let token = EmailVerificationToken { id, email, expires_at, created_at };

        let token = diesel::insert_into(email_verification_token::table)
            .values(&token)
            .on_conflict(email_verification_token::email)
            .do_update()
            .set((
                email_verification_token::id.eq(&token.id),
                email_verification_token::created_at.eq(&token.created_at),
                email_verification_token::expires_at.eq(&token.expires_at),
            ))
            .get_result(&conn)?;

        Ok(token)
    }

    pub fn delete(id: &Vec<u8>) -> Result<usize, ApiError> {
        let conn = db::connection()?;

        let res = diesel::delete(
                email_verification_token::table
                    .filter(email_verification_token::id.eq(id))
            )
            .execute(&conn)?;

        Ok(res)
    }
}

We are using a random generated 32-bit value to make it difficult to guess the token. But for production applications I would also suggest a rate limiter to make brute force attacks more difficult.

We don’t want to have several tokens for each email, so we are using .on_conflict() to handle these conflicts and using that to overwrite the old token when we are creating a new. By using this approach, we also have a way for the user to create a new token in case he deleted the confirmation email by mistake or waited too long to activate the user.

Short generics example

To keep it simple I have avoided to use generics until now, but for our email API we can really make our life a bit easier by using that. Generics allows us to create functions that will accept multiple types for the same parameter, as long as the type has the trait that we need.

We can take our ApiError::new() method as an example. You have probably noticed that we have written a lot of .to_string() inside our ApiError::new() functions, since the message is required to be a String. But what if we let the message be a generic type? We don’t really care as long as we can turn the message into a String, do we?

// src/api_error.rs
impl ApiError {
    pub fn new<T: Into<String>>(status_code: u16, message: T) -> ApiError {
        ApiError { status_code, message: message.into() }
    }
}

This should not break our code since String also implements Into<String>. But what happens now if you remove one of the .to_string() methods inside one of the ApiError::new() menthods, so that we will give a string literal as a parameter instead of a String. It still seems to work since &str also implements Into<String>.

Email API

For calling the Sendinblue API we will need to be able to do an http call. For this we are going to use reqwest. We are also going to need a way to decode and encode our token into something that can be sent with the email and passed back with the json request. For this we can use the hex crate that will help us convert a chunk of bytes into hexadecimal. So let’s add these dependencies.

[dependencies]
hex = "0.4"
reqwest = "0.9"

Now that installed the dependencies we can go ahead and create our email sending API.

// src/email/api.rs
use crate::api_error::ApiError;
use serde::Serialize;
use std::collections::HashMap;

lazy_static::lazy_static! {
    static ref SENDINBLUE_API_KEY: String = std::env::var("SENDINBLUE_API_KEY").unwrap_or("".to_string());
}

#[derive(Debug, Serialize)]
pub struct Contact {
    email: String,
    name: Option<String>,
}

impl Contact {
    pub fn new<T: Into<String>>(email: T, name: T) -> Self {
        Contact { email: email.into(), name: Some(name.into()) }
    }
}

impl<T: Into<String>> From<T> for Contact {
    fn from(email: T) -> Self {
        Contact { email: email.into(), name: None }
    }
}

#[derive(Debug, Serialize)]
pub struct Email {
    sender: Contact,
    #[serde(rename = "to")]
    recipients: Vec<Contact>,
    subject: String,
    #[serde(rename = "htmlContent")]
    html: Option<String>
}

impl Email {
    pub fn new(sender: Contact) -> Self {
        Email {
            sender,
            recipients: Vec::new(),
            subject: "".to_string(),
            html: None,
        }
    }

    pub fn add_recipient<T: Into<Contact>>(mut self, recipient: T) -> Self {
        self.recipients.push(recipient.into());
        self
    }

    pub fn set_subject<T: Into<String>>(mut self, subject: T) -> Self {
        self.subject = subject.into();
        self
    }

    pub fn set_html<T: Into<String>>(mut self, html: T) -> Self {
        self.html = Some(html.into());
        self
    }

    pub fn send(self) -> Result<String, ApiError> {
        let client = reqwest::Client::new();
        let mut response = client.post("https://api.sendinblue.com/v3/smtp/email")
            .header("Accept", "application/json")
            .header("api-key", SENDINBLUE_API_KEY.as_str())
            .json(&self)
            .send()
            .map_err(|e| ApiError::new(500, format!("Failed to send email: {}", e)))?;

        let status = response.status().as_u16();
        let mut body: HashMap<String, String> = response
            .json()
            .map_err(|e| ApiError::new(500, format!("Failed to read sendinblue response: {}", e)))?;

        match status {
            201 => Ok(body.remove("messageId").unwrap_or("".to_string())),
            _ => {
                let message = body.remove("message").unwrap_or("Unknown error".to_string());
                Err(ApiError::new(500, format!("Failed to send email: {}", message)))
            }
        }
    }
}

If you are new to Rust I guess you might stumble a bit on this line: impl<T: Into<String>> From<T> for Contact. It is not too much more complicated than our last example, but let’s break that into a couple of parts that will make it easier to understand.

What if we would have written this line like this: impl From<String> for Contact. This hopefully seems a bit more familiar. Here we are implementing the From trait for Contact so that we can easily convert a String into a Contact. But we also want to do the same for &str. So to avoid repeating ourselves we are defining a generic type <T: Into<String>>, that we can use instead of the String. Now we are sure that whatever type that we are getting, we will be able to turn into a String with the .into() method.

If you have a look at the Sendinblue API documentation, you can see that it does not look exactly like our model. The Sendinblue API says that they want the recipient in the to field and the HTML in the htmlContent field. Serde easily let’s us change that with the #[serde(rename = “name”)] attribute.

In the end we have the send method that will call the Sendinblue API to send our email. If we succeed we will return the message id to the caller.

Invitation and registration endpoint

Now that we have API’s for sending email and creating tokens we can create the endpoints for sending the confirmation email and registration the user.

// src/auth/routes.rs
// ..
use crate::email::{Email, Contact};
use crate::email_verification_token::{EmailVerificationToken, EmailVerificationTokenMessage};
use chrono::Utc;
use hex;
use serde::Deserialize;

#[post("/invite")]
async fn invite(body: web::Json<EmailVerificationTokenMessage>) -> Result<HttpResponse, ApiError> {
    let body = body.into_inner();
    let token = EmailVerificationToken::create(body.clone())?;
    let token_string = hex::encode(token.id);

    Email::new(Contact::new("tore@cloudmaker.dev", "Cloudmaker"))
        .add_recipient(body.email)
        .set_subject("Confirm your email")
        .set_html(format!("Your confirmation code is: {}", &token_string))
        .send()?;

    Ok(HttpResponse::Ok().json(json!({"message": "Verification email sent"})))
}

#[derive(Deserialize)]
struct RegistrationMessage {
    token: String,
    email: String,
    password: String,
}

#[post("/register")]
async fn register(body: web::Json<RegistrationMessage>) -> Result<HttpResponse, ApiError> {
    let body = body.into_inner();
    let token_id = hex::decode(body.token)
        .map_err(|e| ApiError::new(403, "Invalid token"))?;

    let token = EmailVerificationToken::find(&token_id)
        .map_err(|e| {
            match e.status_code {
                404 => ApiError::new(403, "Invalid token"),
                _ => e,
            }
        })?;

    if token.email != body.email {
        return Err(ApiError::new(403, "Invalid token"));
    }

    if token.expires_at < Utc::now().naive_utc() {
        return Err(ApiError::new(403, "Token expired"));
    }

    let user = User::create(UserMessage { email: body.email, password: body.password })?;

    Ok(HttpResponse::Ok().json(json!({"message": "Successfully registered", "user": user})))
}

// ..

pub fn init_routes(cfg: &mut web::ServiceConfig) {
    cfg.service(invite);
    cfg.service(register);
    // ..
}

For our invite endpoint we are just creating a token, encoding it and sending it to via email. At best we should have had a frontend where we could have directed the user, so he could just click a link instead of having to copy and paste the activation token.

I will not create a frontend for now, since I still have more topics I want to cover about Rust first. But in the future I might be creating a frontend for our app if there is any interest. In that case I would probably be using elm, since that aims to reliable, fast and easy to refactor, just like Rust.

For the registration endpoint we are just searching for the token and validating that it matches the email and that it is not expired. To not reveal too much information to an attacker, we will just let the user know that the token is invalid if not everything is correct. The only exception is if the token is expired, since that can be helpful information for the user.

Now let’s give it a try. First use the invite endpoint to send yourself the confirmation email. Then register with the token you received in the email.

The complete code example for this tutorial is available on github.

Rust Web Development Tutorial: Authentication

Tore Pettersen — Sat, 25 Jan 2020 09:07:39 +0000

Originaly posted on cloudmaker.dev

In this tutorial, we are going to create Authentication for the REST API that we made in the last tutorial, so I highly recommend reading that first. Alternatively, you can clone the code from the last tutorial on github.
We will be using Redis for handling our sessions, so if you don’t have Redis installed on your computer, you should do that first.

Password hashing

You should never store real user passwords in the database unencrypted. So let’s start with encrypting the passwords. For hashing the passwords we going to use Argon2. Argon2 is known for being a secure hashing algorithm and even won the Password Hashing Competition (PHC). We are also going to need a random generator for creating a salt. So let’s add the dependencies to our Cargo.toml.

[dependencies]
rand = "0.7"
rust-argon2 = "0.5"

We are going to hash the password as part of our User creation API and while we are at it we can create a function for verifying the password.

// src/user/model.rs
use argon2::Config;
use rand::Rng;
// ...

impl User {
    // ...
    pub fn hash_password(&mut self) -> Result<(), ApiError> {
        let salt: [u8; 32] = rand::thread_rng().gen();
        let config = Config::default();

        self.password = argon2::hash_encoded(self.password.as_bytes(), &salt, &config)
            .map_err(|e| ApiError::new(500, format!("Failed to hash password: {}", e)))?;

        Ok(())
    }

    pub fn verify_password(&self, password: &[u8]) -> Result<bool, ApiError> {
        argon2::verify_encoded(&self.password, password)
            .map_err(|e| ApiError::new(500, format!("Failed to verify password: {}", e)))
    }
}
// ...

For creating the hash you can see that we are using a random 32-bit salt for each password. This is to avoid that users that have the same password ends up having the same password hash and it also makes it harder for an attacker to crack the passwords.

It not good a practice to send the password hash back to the user. Serde makes it quite easy to hide the password hash from the user, by skipping serialization of the password.

// src/user/model.rs
#[derive(Serialize, Deserialize, Queryable, Insertable)]
#[table_name = "user"]
pub struct User {
    pub id: Uuid,
    pub email: String,
    #[serde(skip_serializing)]
    pub password: String,
    pub created_at: NaiveDateTime,
    pub updated_at: Option<NaiveDateTime>,
}

We can now try to create a new user over with our user create endpoint and we will see that the user password is encrypted in our database.

Register user

Registering a user is going to be a simple task. We technically already have an endpoint for that. But let’s make the registration a part of our new auth module. So let’s make the auth module with a registration endpoint.

// src/auth/routes.rs
use crate::api_error::ApiError;
use crate::user::{User, UserMessage};
use actix_web::{post, get, web, HttpResponse};

#[post("/register")]
async fn register(user: web::Json<UserMessage>) -> Result<HttpResponse, ApiError> {
    let user = User::create(user.into_inner())?;
    Ok(HttpResponse::Ok().json(user))
}

pub fn init_routes(cfg: &mut web::ServiceConfig) {
    cfg.service(register);
}

We also need to make the init method for the routes available outside of the module.

// src/auth/mod.rs
mod routes;

pub use routes::init_routes;

Then we can connect our new module with the rest of our application.

// src/main.rs
// ...
mod auth;

#[actix_rt::main]
async fn main() -> std::io::Result<()> {
    // ...
    let mut server = HttpServer::new(|| 
        App::new()
            .configure(auth::init_routes)
    );
    // ...
}

Now our new registration endpoint should work in the same fashion as our user create endpoint worked.

Sign in and out

I have found quite a lot of tutorials suggestion to use json web tokens for handling sessions and that they are supposed to be more secure than storing the session data in a cookie. But where are you going to store your token? Maybe in a cookie? By doing that we can still be a subject of a CSRF attack, which I have heard that many people say that you can avoid by using json web tokens. The other alternative is to store the token in the session storage. Now you might be secure against CSRF attacks, but you could now be exposed to XSS attacks which could be even more dangerous.

A more secure way to store session data is on the server-side in a key-value store like Redis. We can still be a subject of CSRF attacks since we need to store our session key in a cookie. So make sure to protect yourself against these kinds of attacks before you deploy your application to production. CSRF attacks are a more complex topic, so I will be covering that more in-depth in a separate blog post in the future.

For handling sessions, we are going to need another couple of dependencies.

[dependencies]
actix-redis = { version = "0.8", features = ["web"] }
actix-session = "0.3"

To let our application know where to find Redis we are going to add another couple of environment variables.

REDIS_HOST=127.0.0.1
REDIS_PORT=6379

We also have to set up the middleware that will handle the sessions for us.

// src/main.rs
// ...
use actix_redis::RedisSession;

#[actix_rt::main]
async fn main() -> std::io::Result<()> {
    // ...
    let redis_port = env::var("REDIS_PORT").expect("Redis port not set");
    let redis_host = env::var("REDIS_HOST").expect("Redis host not set");

    let mut server = HttpServer::new(move|| 
        App::new()
            .wrap(RedisSession::new(format!("{}:{}", redis_host, redis_port), &[0; 32]))
            .configure(auth::init_routes)
    );
    // ...
}

For signing in, we need to be able to find a user only by knowing the email. One way to find the user by email is by extending our user API with a new method.

// src/user/model.rs
// ...
impl User {
    pub fn find_by_email(email: String) -> Result<Self, ApiError> {
        let conn = db::connection()?;

        let user = user::table
            .filter(user::email.eq(email))
            .first(&conn)?;

        Ok(user)
    }
    // ...
}

Also for convenience, we are going to make a default way for handling Actix errors. This way we can use the question mark to handle these kinds of errors instead of explicitly having to map each of these errors to ApiError.

// src/api_error.rs
use actix_web::error::Error as ActixError;
// ...

impl From<ActixError> for ApiError {
    fn from(error: ActixError) -> ApiError {
        ApiError::new(500, error.to_string())
    }
}

Now we can create the endpoints for signing in and out.

// src/auth/routes.rs
use actix_session::Session;
use serde_json::json;
use uuid::Uuid;
// ...

#[post("/sign-in")]
async fn sign_in(credentials: web::Json<UserMessage>, session: Session) -> Result<HttpResponse, ApiError> {
    let credentials = credentials.into_inner();

    let user = User::find_by_email(credentials.email)
        .map_err(|e| {
            match e.status_code {
                404 => ApiError::new(401, "Credentials not valid!".to_string()),
                _ => e,
            }
        })?;

    let is_valid = user.verify_password(credentials.password.as_bytes())?;

    if is_valid == true {
        session.set("user_id", user.id)?;
        session.renew();

        Ok(HttpResponse::Ok().json(user))
    }
    else {
        Err(ApiError::new(401, "Credentials not valid!".to_string()))
    }
}

#[post("/sign-out")]
async fn sign_out(session: Session) -> Result<HttpResponse, ApiError> {
    let id: Option<Uuid> = session.get("user_id")?;

    if let Some(_) = id {
        session.purge();
        Ok(HttpResponse::Ok().json(json!({ "message": "Successfully signed out" })))
    }
    else {
        Err(ApiError::new(401, "Unauthorized".to_string()))
    }
}

pub fn init_routes(cfg: &mut web::ServiceConfig) {
    cfg.service(register);
    cfg.service(sign_in);
    cfg.service(sign_out);
}

If we now hit our sign-in endpoint we should be able to see the session key as a part of the cookie, if we have already registered the user. We can also try sign-out, and we will then see that the cookie will be deleted. I used Insomnia for testing it.

Using the session data

Now let’s use the session data to get out the user information. So let’s create an endpoint that will give us the user data for the signed-in user.

// src/auth/routes.rs
// ...
#[get("/who-am-i")]
async fn who_am_i(session: Session) -> Result<HttpResponse, ApiError> {
    let id: Option<Uuid> = session.get("user_id")?;

    if let Some(id) = id {
        let user = User::find(id)?;
        Ok(HttpResponse::Ok().json(user))
    }
    else {
        Err(ApiError::new(401, "Unauthorized".to_string()))
    }
}

pub fn init_routes(cfg: &mut web::ServiceConfig) {
    cfg.service(register);
    cfg.service(sign_in);
    cfg.service(sign_out);
    cfg.service(who_am_i);
}

Now if we hit the newly created endpoint we should get the user data if we are signed in and a status 401 if we are not.

The complete source code for this tutorial is available on github in case you need it.

Rust Web Development Tutorial: REST API

Tore Pettersen — Sat, 11 Jan 2020 08:53:43 +0000

Originaly posted on cloudmaker.dev

In this tutorial, we are going to create a REST API in Rust with Actix web 2.0 and Diesel. We will be using Postgres as our database, so if you don’t have Postgres installed on your computer, you should do that first.

Hello world

We are going to start by creating our project with Cargo and move into the project directory.

$ cargo new rest_api
$ cd rest_api

We need to add Actix web to our dependencies for our first example. So let’s add that to the Cargo.toml.

[dependencies]
actix-web = "2.0"
actix-rt = "1.0"

And then we set up the request handler and server in src/main.rs.

// src/main.rs
use actix_web::{App, HttpResponse, HttpServer, Responder, get};

#[get("/")]
async fn index() -> impl Responder {
    HttpResponse::Ok().body("Hello world!")
}

#[actix_rt::main]
async fn main() -> std::io::Result<()> {
    HttpServer::new(|| {
        App::new()
            .service(index)
    })
        .bind("127.0.0.1:5000")?
        .run()
        .await
}

Now that we have created our first server, let’s run it with cargo run. To test our REST API, let’s visit localhost:5000, and we should hopefully see our hello world.

Auto Reloading

It could be quite tedious to recompile the code manually every time that we make a change, so let’s have cargo-watch recompile it for us on every change. It would also make sense to combine it with the listenfd crate and the systemfd utility to keep the connection open while our code recompiles. This way we avoid that our REST client breaks off the requests while the code recompiles since it can’t reach the server. By keeping the connection open we could just make a call to the server and the server will respond as soon as it has recompiled and is ready to handle our request.

For this, we need to install cargo-watch and systemfd. Both are written in Rust and available on crates.io, so we can install them with cargo.

$ cargo install systemfd cargo-watch

We also need to add listenfd to our dependencies.

[dependencies]
listenfd = "0.3"

Then we need to make some changes to src/main.rs so that we can use the listener that is provided for us by systemfd, but also have a fallback for cases when we don’t need it. Like when we are deploying our code.

// src/main.rs
use actix_web::{get, App, HttpResponse, HttpServer, Responder};
use listenfd::ListenFd;

#[get("/")]
async fn index() -> impl Responder {
    HttpResponse::Ok().body("Hello world!")
}

#[actix_rt::main]
async fn main() -> std::io::Result<()> {
    let mut listenfd = ListenFd::from_env();
    let mut server = HttpServer::new(||
        App::new()
            .service(index)
    );

    server = match listenfd.take_tcp_listener(0)? {
        Some(listener) => server.listen(listener)?,
        None => server.bind("127.0.0.1:5000")?,
    };

    server.run().await
}

Now we can run the server and file watcher that will automatically recompile our code on changes with this command.

$ systemfd --no-pid -s http::5000 -- cargo watch -x run

Environment variables and logging

You would probably deploy your code at some point. Then you might want to run the server with some different settings than on your local machine, like using a different port or a different level of logging. You might also need to use some secrets that should not be part of your code, like database credentials. For this we could use environment variables.

Also when you deploy your code you could be sure that it will run into problems at some point. And to help with solving these problems it is important with good logging, so that we could figure out what went wrong and solve the problem.

For setting up environment variables and logging we are going to add another few dependencies.

[dependencies]
dotenv = "0.11"
log = "0.4"
env_logger = "0.6"

For convenience let’s set up some default parameters that we could use for development. We could do that by creating a .env file in the root of our project.

RUST_LOG=rest_api=info,actix=info
HOST=127.0.0.1
PORT=5000

The log crate provides five different logs levels which are error, warn, info, debug and trace, where error represents the highest-priority log messages and trace the lowest. For this tutorial we will set the log level to info for our REST API and Actix, meaning we will get all messages from error, warn and info.

To activate logging and environment variable we would only need to make a few small changes to our main file.

// src/main.rs
#[macro_use]
extern crate log;

use actix_web::{get, App, HttpResponse, HttpServer, Responder};
use dotenv::dotenv;
use listenfd::ListenFd;
use std::env;

#[get("/")]
async fn index() -> impl Responder {
    HttpResponse::Ok().body("Hello world!")
}

#[actix_rt::main]
async fn main() -> std::io::Result<()> {
    dotenv().ok();
    env_logger::init();

    let mut listenfd = ListenFd::from_env();
    let mut server = HttpServer::new(||
        App::new()
            .service(index)
    );

    server = match listenfd.take_tcp_listener(0)? {
        Some(listener) => server.listen(listener)?,
        None => {
            let host = env::var("HOST").expect("Host not set");
            let port = env::var("PORT").expect("Port not set");
            server.bind(format!("{}:{}", host, port))?
        }
    };

    info!("Starting server");
    server.run().await
}

The dotenv().ok() function will grab the environment variables from the .env file and add them to our servers environment variables. This way we could use these variables by using the std::env::var() function, as we have done for setting the host and port.

The log crate will provide five micros that we could use for writing log messages. One for each log level: error!, warn!, info! debug! and trace!. To see our log messages in stdout or stderr we need to initiate the env_logger which we do with a single function: env_logger::init().

Api endpoints

Our API will be sending and receiving json data, so we need a way to Serialize and Deserialize json into a data structure recognized by Rust. For this we are going to use Serde. So we need to add that to our list of dependencies.

[dependencies]
serde = "1.0"
serde_json = "1.0"

Now we will define a user model and we will add the Serialize and Deserialize annotations so that our model can be extracted from and converted to json.

// src/user/model.rs
use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize)]
pub struct User {
    pub id: i32,
    pub email: String,
}

Let’s go ahead and create our REST API’s. Our next step will be to persist the data, but for now we will just be using hard coded dummy data.

// src/user/routes.rs
use crate::user::User;
use actix_web::{get, post, put, delete, web, HttpResponse, Responder};
use serde_json::json;

#[get("/users")]
async fn find_all() -> impl Responder {
    HttpResponse::Ok().json(
        vec![
            User { id: 1, email: "tore@cloudmaker.dev".to_string() },
            User { id: 2, email: "tore@cloudmaker.dev".to_string() },
        ]
    )
}

#[get("/users/{id}")]
async fn find() -> impl Responder {
    HttpResponse::Ok().json(
        User { id: 1, email: "tore@cloudmaker.dev".to_string() }
    )
}

#[post("/users")]
async fn create(user: web::Json<User>) -> impl Responder {
    HttpResponse::Created().json(user.into_inner())
}

#[put("/users/{id}")]
async fn update(user: web::Json<User>) -> impl Responder {
    HttpResponse::Ok().json(user.into_inner())
}

#[delete("/users/{id}")]
async fn delete() -> impl Responder {
    HttpResponse::Ok().json(json!({"message": "Deleted"}))
}

pub fn init_routes(cfg: &mut web::ServiceConfig) {
    cfg.service(find_all);
    cfg.service(find);
    cfg.service(create);
    cfg.service(update);
    cfg.service(delete);
}

We also need to connect the user routes with the user model and make it available outside of the user directory.

// src/user/mod.rs
mod model;
mod routes;

pub use model::User;
pub use routes::init_routes;

We can now replace our “Hello world” endpoint with the actual user endpoints.

// src/main.rs
#[macro_use]
extern crate log;

use actix_web::{App, HttpServer};
use dotenv::dotenv;
use listenfd::ListenFd;
use std::env;

mod user;

#[actix_rt::main]
async fn main() -> std::io::Result<()> {
    dotenv().ok();
    env_logger::init();

    let mut listenfd = ListenFd::from_env();
    let mut server = HttpServer::new(|| 
        App::new()
            .configure(user::init_routes)
    );

    server = match listenfd.take_tcp_listener(0)? {
        Some(listener) => server.listen(listener)?,
        None => {
            let host = env::var("HOST").expect("Host not set");
            let port = env::var("PORT").expect("Port not set");
            server.bind(format!("{}:{}", host, port))?
        }
    };

    info!("Starting server");
    server.run().await
}

We should now be able to test the user endpoints we just created by calling them. You can f.eks use Insomnia or curl for that.

Persisting data

Having a few endpoints is not really helpful if we are not able to persist the data. For this we are going to use Diesel, which is a quite a mature ORM. Diesel will let us connect to Postgres, MySQL and SQLite, but in this tutorial we will only be covering Postgres.

Diesel depends on openssl and libpq, so we need to install those before we can install Diesel CLI. If you are using a Debian like OS you could simply install that using apt.

$ sudo apt install openssl libpq-dev -y

When we have installed the needed dependencies we can install Diesel CLI.

$ cargo install diesel_cli --no-default-features --features postgres

To let Diesel know where our database is, we need to add DATABASE_URL to our .env file.

DATABASE_URL=postgres://postgres:password@localhost/rest_api

We can use Diesel CLI to set up Diesel in our project, and create files for our user migration.

$ diesel setup
$ diesel migration generate create_user

In the migrations folder we should now be able to find a folder for our first migration. This folder should contain two files. One named up.sql where we will be creating our user schema and one named down.sql which should revert everything we did in the up.sql file.

-- up.sql
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";

CREATE TABLE "user" (
    id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
    email TEXT UNIQUE NOT NULL,
    password TEXT NOT NULL,
    created_at TIMESTAMP   NOT NULL DEFAULT current_timestamp,
    updated_at TIMESTAMP
);

-- down.sql
DROP TABLE "user";

Now that we have created our first migration, we can run it with Diesel CLI.

$ diesel migration run

This command should also create a schema file that we will use later for building sql queries. Default location for this file is src/schema.rs.

When we are dealing with databases we should be prepared for that problems that can occur like connection issues or database conflicts. So we are going to create an own error type to handle these problems.

// src/api_error.rs
use actix_web::http::StatusCode;
use actix_web::{HttpResponse, ResponseError};
use diesel::result::Error as DieselError;
use serde::Deserialize;
use serde_json::json;
use std::fmt;

#[derive(Debug, Deserialize)]
pub struct ApiError {
    pub status_code: u16,
    pub message: String,
}

impl ApiError {
    pub fn new(status_code: u16, message: String) -> ApiError {
        ApiError { status_code, message }
    }
}

impl fmt::Display for ApiError {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        f.write_str(self.message.as_str())
    }
}

impl From<DieselError> for ApiError {
    fn from(error: DieselError) -> ApiError {
        match error {
            DieselError::DatabaseError(_, err) => ApiError::new(409, err.message().to_string()),
            DieselError::NotFound => ApiError::new(404, "Record not found".to_string()),
            err => ApiError::new(500, format!("Diesel error: {}", err)),
        }
    }
}

impl ResponseError for ApiError {
    fn error_response(&self) -> HttpResponse {
        let status_code = match StatusCode::from_u16(self.status_code) {
            Ok(status_code) => status_code,
            Err(_) => StatusCode::INTERNAL_SERVER_ERROR,
        };

        let message = match status_code.as_u16() < 500 {
            true => self.message.clone(),
            false => {
                error!("{}", self.message);
                "Internal server error".to_string()
            },
        };

        HttpResponse::build(status_code)
            .json(json!({ "message": message }))
    }
}

Our error type consists of a status code and a message that we will use to create the error message. We create the error message by implementing ResponseError, which we are using to create a json response.

In case we have an internal server error, it is probably not the best show the users what went wrong. For this case we will just let the user know that something went wrong and write the error message to the logs.

Our error type also implements From<diesel::result::Error>, so that we don’t have to do that for each time we have to handle a Diesel error.

[dependencies]
chrono = { version = "0.4", features = ["serde"] }
diesel = { version = "1.4", features = ["postgres", "r2d2", "uuid", "chrono"] }
diesel_migrations = "1.4"
lazy_static = "1.4"
r2d2 = "0.8"
uuid = { version = "0.6", features = ["serde", "v4"] }

For handling state we will be using statics, although Actix has built in state management. You could read my post on loose coupling to understand why I decided to go for this approach, although some people might disagree with it.

Now let’s establish a database connection and use r2d2 to efficiently handle the connection pool.

// src/db.rs
use crate::api_error::ApiError;
use diesel::pg::PgConnection;
use diesel::r2d2::ConnectionManager;
use lazy_static::lazy_static;
use r2d2;
use std::env;

type Pool = r2d2::Pool<ConnectionManager<PgConnection>>;
pub type DbConnection = r2d2::PooledConnection<ConnectionManager<PgConnection>>;

embed_migrations!();

lazy_static! {
    static ref POOL: Pool = {
        let db_url = env::var("DATABASE_URL").expect("Database url not set");
        let manager = ConnectionManager::<PgConnection>::new(db_url);
        Pool::new(manager).expect("Failed to create db pool")
    };
}

pub fn init() {
    info!("Initializing DB");
    lazy_static::initialize(&POOL);
    let conn = connection().expect("Failed to get db connection");
    embedded_migrations::run(&conn).unwrap();
}

pub fn connection() -> Result<DbConnection, ApiError> {
    POOL.get()
        .map_err(|e| ApiError::new(500, format!("Failed getting db connection: {}", e)))
}

With the database connection established we can finally create the API for creating, reading, updating and deleting the user data.

// src/user/model.rs
use crate::api_error::ApiError;
use crate::db;
use crate::schema::user;
use chrono::{NaiveDateTime, Utc};
use diesel::prelude::*;
use serde::{Deserialize, Serialize};
use uuid::Uuid;

#[derive(Serialize, Deserialize, AsChangeset)]
#[table_name = "user"]
pub struct UserMessage {
    pub email: String,
    pub password: String,
}

#[derive(Serialize, Deserialize, Queryable, Insertable)]
#[table_name = "user"]
pub struct User {
    pub id: Uuid,
    pub email: String,
    pub password: String,
    pub created_at: NaiveDateTime,
    pub updated_at: Option<NaiveDateTime>,
}

impl User {
    pub fn find_all() -> Result<Vec<Self>, ApiError> {
        let conn = db::connection()?;

        let users = user::table
            .load::<User>(&conn)?;

        Ok(users)
    }

    pub fn find(id: Uuid) -> Result<Self, ApiError> {
        let conn = db::connection()?;

        let user = user::table
            .filter(user::id.eq(id))
            .first(&conn)?;

        Ok(user)
    }

    pub fn create(user: UserMessage) -> Result<Self, ApiError> {
        let conn = db::connection()?;

        let user = User::from(user);
        let user = diesel::insert_into(user::table)
            .values(user)
            .get_result(&conn)?;

        Ok(user)
    }

    pub fn update(id: Uuid, user: UserMessage) -> Result<Self, ApiError> {
        let conn = db::connection()?;

        let user = diesel::update(user::table)
            .filter(user::id.eq(id))
            .set(user)
            .get_result(&conn)?;

        Ok(user)
    }

    pub fn delete(id: Uuid) -> Result<usize, ApiError> {
        let conn = db::connection()?;

        let res = diesel::delete(
                user::table
                    .filter(user::id.eq(id))
            )
            .execute(&conn)?;

        Ok(res)
    }
}

impl From<UserMessage> for User {
    fn from(user: UserMessage) -> Self {
        User {
            id: Uuid::new_v4(),
            email: user.email,
            password: user.password,
            created_at: Utc::now().naive_utc(),
            updated_at: None,
        }
    }
}

And with the user API in place we could use that instead of the fake data we used earlier.

// src/user/routes.rs
use crate::api_error::ApiError;
use crate::user::{User, UserMessage};
use actix_web::{delete, get, post, put, web, HttpResponse};
use serde_json::json;
use uuid::Uuid;

#[get("/users")]
async fn find_all() -> Result<HttpResponse, ApiError> {
    let users = User::find_all()?;
    Ok(HttpResponse::Ok().json(users))
}

#[get("/users/{id}")]
async fn find(id: web::Path<Uuid>) -> Result<HttpResponse, ApiError> {
    let user = User::find(id.into_inner())?;
    Ok(HttpResponse::Ok().json(user))
}

#[post("/users")]
async fn create(user: web::Json<UserMessage>) -> Result<HttpResponse, ApiError> {
    let user = User::create(user.into_inner())?;
    Ok(HttpResponse::Ok().json(user))
}

#[put("/users/{id}")]
async fn update(id: web::Path<Uuid>, user: web::Json<UserMessage>) -> Result<HttpResponse, ApiError> {
    let user = User::update(id.into_inner(), user.into_inner())?;
    Ok(HttpResponse::Ok().json(user))
}

#[delete("/users/{id}")]
async fn delete(id: web::Path<Uuid>) -> Result<HttpResponse, ApiError> {
    let num_deleted = User::delete(id.into_inner())?;
    Ok(HttpResponse::Ok().json(json!({ "deleted": num_deleted })))
}

pub fn init_routes(cfg: &mut web::ServiceConfig) {
    cfg.service(find_all);
    cfg.service(find);
    cfg.service(create);
    cfg.service(update);
    cfg.service(delete);
}

Now we just have to add db, schema and api_error models in our main file. I also strongly recommend to initiate the database, although that is not strictly necessary. We are using lazy_static to handle the database pool. So if we don’t initiate it right way, it won’t be initiated before it will be used. Which then will be when the first user tires to call our API.

// src/main.rs
#[macro_use]
extern crate log;
#[macro_use]
extern crate diesel;
#[macro_use]
extern crate diesel_migrations;

use actix_web::{App, HttpServer};
use dotenv::dotenv;
use listenfd::ListenFd;
use std::env;

mod api_error;
mod db;
mod schema;
mod user;

#[actix_rt::main]
async fn main() -> std::io::Result<()> {
    dotenv().ok();
    env_logger::init();

    db::init();

    let mut listenfd = ListenFd::from_env();
    let mut server = HttpServer::new(|| 
        App::new()
            .configure(user::init_routes)
    );

    server = match listenfd.take_tcp_listener(0)? {
        Some(listener) => server.listen(listener)?,
        None => {
            let host = env::var("HOST").expect("Host not set");
            let port = env::var("PORT").expect("Port not set");
            server.bind(format!("{}:{}", host, port))?
        }
    };

    info!("Starting server");
    server.run().await
}

We should now be able to create, read, update and delete users via the endpoints.

In case you need it you can also find the complete code on github.

Why Should You Care About Loose Coupling?

Tore Pettersen — Tue, 31 Dec 2019 09:53:43 +0000

Originaly posted on cloudmaker.dev

Have you ever been in a situation where a change in your application has caused a strange bug in a completely different part of your application? Or have you at least experienced an API breaking?

You can reduce the chance of these problems by having different parts of your application know as little as possible about each other as possible. This an architectural design called loose coupling.

A loosely coupled application is also a lot easier to split into smaller services if you ever need it. This will be helpful if you ever consider splitting your monolith into microservices.

I have found it a bit hard to find some good practical examples on how to build a loosely coupled application. So here are a few tips you can follow to make your application more loosely coupled.

Folder structure

Imagine you have a folder structure like this

.
├── routes
|   ├──order.rs
|   └── user.rs
├── events
|   ├──order.rs
|   └── user.js
├── models
|   ├── order.rs
|   └── user.rs

With a folder structure like this, you need to open several folders to understand how each of the features works. And when the list of features increases, I find it hard to navigate, since I am forced to have several folders open at once. Although most of it is irrelevant at the given moment since I am usually working on only one feature at a time.

And also if we ever want to split this into several services, we have to sort through several different folders to connect the parts we need.

An alternative way is to group the folders by feature, like this.

.
├── order
|   ├──mod.rs
|   ├──routes.rs
|   ├──events.rs
|   └── model.rs
├── user
|   ├──mod.rs
|   ├──routes.rs
|   ├──events.rs
|   └── model.rs

With a structure like this, it is a lot easier to get an overview of how each feature is working. Here we only need to open one small folder for each feature we are working on. And navigating is also easier since the files that belong together are placed together.

Interaction between components

What happens if you let different parts of your application modify the user data in the database? It might work, but in general, the is not a good idea since you would end up with two different approaches that can cause conflict for each other.

Let’s say the one approach has built-in caching, and the other does not care since it was just a quick and dirty implementation. If we now use the second approach to update the database, we will then end up with an invalid cache.

Having different parts of the application changing data in the same database will also make it really hard to get an overview of who is making these changes. And if you don’t know who is making these changes, it will also be hard to debug the code when you find a bug. Also splitting a monolith into microservices is close to impossible when you don’t have an easy overview of which parts are linked together.

If we rather let the different parts of our application interact through APIs, it will be a lot easier to understand how the different parts are linked together. We could even see all the places where the API is used by searching for a reference to it. Another advantage is that we could rewrite the API, without affecting the parts where it is used. This could f.eks could be helpful if you figure that an API is heavily used and needs to add caching.

So how does a good API look like? The best tip I can give is just to keep it simple. If you f.eks want to create an API for finding a user, it should be enough just to pass the id.

#[get("/users/{id}")]
fn find(id: web::Path<Uuid>) -> Result<HttpResponse, ApiError> {
    let user = User::find(id.into_inner())?;
    Ok(HttpResponse::Ok().json(user))
}

I have seen quite a lot of examples in the rust community where the API connection needs to be passed to the API like this.

#[get("/users/{id}")]
fn find(conn: web::Data<PgConnection>, id: web::Path<Uuid>) -> Result<HttpResponse, ApiError> {
    let user = User::find(conn.into_inner(), id.into_inner())?;
    Ok(HttpResponse::Ok().json(user))
}

Right away you will probably not have any problems going for this approach. But you will probably at some time want to do any changes to this API, like adding a cache or maybe publish or subscribe to an event queue.

By adding or removing parameters to our API we will make it break. And if we want to solve that, we have to add this new parameter every single place this API is used. In small projects that might be doable, but what about bigger projects which are maintained by several developers?

Moving towards Microservices

I don’t suggest starting out with a microservice architecture since that will cause quite a lot of overhead to your project. But if you at some point ever get a lot of customers you might have to scale it up. One way is to divide your application into several smaller services so that each service has a smaller load.

If you followed the tips above you will have a lot easier time to split the different parts of your application into smaller services. You can easily do that by just drag and drop the folders for each feature into your new service.

But wait, this will break our APIs don’t it? Yes, it will, but it is actually a quite easy solution for fixing that since all the users of our APIs don’t make any assumptions on how the API is working. This means that we could just rewrite our APIs to make a remote procedure calls to our newly created service, which will handle the request. And the user of the API doesn’t have to do any changes since the API still looks the same.

7 Reasons Why I Chose Rust

Tore Pettersen — Sat, 14 Dec 2019 09:13:14 +0000

Originaly posted on cloudmaker.dev

For most programming languages I feel you have to sacrifice either safety and convenience or performance.

You have languages like C and C++ which is incredibly fast since they don’t have any runtime or garbage collector. But for that, you have to sacrifice quite a lot of effort to write the code and you can easily end up with bugs that are quite hard to catch.

On the other side, you have languages like Python and Ruby which are easy and fast to write since they handle a lot of the hard stuff in the background like memory management. But for that, you have to sacrifice quite a lot of runtime performance.

Go is definitely one of my favorite programming languages and I feel that it has done a quite good attempt to manage both, although it does not quite reach up to the raw performance of C and C++. But there are a few things I dislike about go, like the way error handling is handled and the missing package manager.

And then we have Rust which aims to be both performant, reliable and productive. And I believe these goals have quite a lot to do with the hype around Rust right now. Rust is even the most loved programming language according to the stack overflow survey. But why is Rust so loved by programmers if there is no garbage collector to help you out with memory management and no runtime to help out with multithreading?

Rust is performant

Rust compiles into native machine code and can run on multiple platforms. It is even a great fit for embedded systems. Native machine code with no overhead in the form of a garbage collector or runtime makes Rust binaries so performant that they are even comparable with C and C++.

Rust is reliable

Instead of a garbage collector, Rust has quite strict rules that you have to follow that guarantee memory and thread-safety. If you are not following these rules the compiler won’t even let you compile the code. The compiler is also enforcing you to handle all possible errors and return values right away, so bad code will never make it to production. But don’t worry, the compiler is quite helpful in pointing out your mistakes, so it is quite easy for you to fix them before your colleagues will point them out to you during the code review.

Rust helps you be productive

You will not be able to write a prototype as fast in Rust as you can in JavaScript or Python. Writing Rust code initially takes a bit longer, but I feel I save quite a lot of time by having the compiler point out silly mistakes right away. At least compared to having to run my program just to see it fail, or even worse, having a customer or project manager finding it for you in production. I definitely prefer having the compiler pointing me to the exact line number where I made the mistake than trying to get a project manager explaining to me the steps he made to produce the bug, so I can reproduce it.

Another big timesaver is refactoring. The compiler will once again help you out if you break some existing code. I have done some bigger changes to an API, and all I had to do was just to follow the error messages from the compiler to find all the places it broke. When I was through the error messages I could see the program run as expected again.

Powerful language features

Rust has a lot of the more powerful language features we see in modern programming languages like pattern matching, generics, and traits. Rust also has one of the more powerful macro features I have seen and is really great for code reuse, even in more complex situations where a function call won’t be possible. And of course, no runtime overhead since they are converted into actual code at compile time. The usage of monadic error handling in Rust makes it really easy to make sure that you have handled all possible return values without being a pain. And if you are a fan of functional programming Rust also has some functions like map(), filter() and find().

Cargo

The package manager for Rust is called Cargo and it does more than just grabbing the packages for you. Cargo will help you with everything from creating your project, to building your code. You can also use it to handle your dependencies, testing your code and upload your packages to crates.io, which is the packages registry for Rust. And best of all, most of it works straight out of the box. No need to write a huge make file or learn a tool like Webpack. You mostly just have to list your dependencies and Cargo will take care of the rest.

Helpful community

I got to admit that Rust is one of the more difficult languages I have learned. Rust really has some more difficult concepts you have to learn, which is not common in other programming languages. But to help you, there is actually an official book that you could read for free, and it is great. If you are of the type that prefers physical books you could also grab a hard copy. Rust also has the most polite and helpful community I have come across. So if you ever get stuck, don’t worry about getting help from the community.

Libraries

There are about 30 000 libraries available on crates.io, at the moment of writing. That might sound like a small number compared to the 1 000 000 libraries on NPM, but I still feel that I can find what I need. And also I find that the quality of the libraries usually is better than what I am used to from JavaScript. You have libraries like the Actix web which is an incredibly performant web framework. Just check out the benchmarks. The serialization framework Serde is also top of the class and is even faster than comparable libraries in C++. And Diesel is such a good ORM tool, that it is even more popular than the native client libraries for each database.

So where should you start?

So hopefully you are now all convinced that Rust is a good choice for you as well. Or did I forget to mention that Rust is also quite fun to write as well? Anyway, a great place to start is by reading the book online or in book form. I have also just made a blog where you can find a few tutorials.