DEV Community: Martin PAUCOT

Roll your own auth with Rust and Protobuf

Martin PAUCOT — Sat, 28 Oct 2023 16:25:50 +0000

Introduction
- 1. What is Protobuf?
- 2. What are we going to accomplish
- 3. Requirements
Get started
- 1. Create the PostgreSQL database
- 2. Create the Protobuf definitions
- 3. Setup Buf to generate SDKs
- 4. Setup the Rust project
- 5. Use Diesel to manage our users
- 6. Implement the Tokio Services
You made it 🚀

Introduction

Before starting, take a deep breath, maybe 5 minutes of meditation and prepare a drink because this tutorial imply Rust code. Not complex, but Rust code.

But first, let's talk about Protobuf, what is it?

1. What is Protobuf?

According to the documentation:

"Protocol Buffers are language-neutral, platform-neutral extensible mechanisms for serializing structured data."

JSON is really flexible when you want to share data through services you can decode JSON without knowing it's structure first.
But it is unstructured, takes a lot of space and bandwith.

With Protocol Buffers you define a message and its structure that must be known by both the server and the client to encode and decode.

// user.proto

syntax = "proto3";

message User {
    string firstname = 1;
    string lastname = 2;
    string email = 3;
}

You can then use generators to create the SDKs for your favorites languages. You can generate a Javascript one for your frontend and a Rust one for your backend.

If you are using Remote Procedure Call (RPC) like gRPC, you can leverage the features of Protobuf and the generators to automatically generate interfaces and code for both your client and server SDKs.
The only thing you have to do next, is implement the methods of your services.

// user.proto

syntax = "proto3";

message LoginRequest {
    string email = 1;
    string password = 2;
}

service Auth {
    rpc Login (LoginRequest) returns (User);
}

2. What are we going to accomplish

Create a PostgreSQL database using Docker and Docker compose
Create Protobuf definitions and use Buf to generate the SDKs
Setup a gRPC server in Rust using Tonic
Create a JWT authentication system with Diesel and Tonic

3. Requirements

You must have a basic understanding of Rust, I will not deep dive into how to write Rust. As I am myself, a Rust beginner. But I encourage you to check the different crates documentation to have a better understanding on how everything works.

You must know how JWT works as I will not explain it in this blog post.

You must have all the required tools installed:

If you are lost somewhere, or you directly want to go straigth to the code, the repository is available here https://github.com/kerwanp/rust-proto-demo.

Get started

1. Create the PostgreSQL database

As we want to persist our users we need a database. We are going to use PostgreSQL.

For simplicity of use, we will use Docker Compose to manage it. Make sure you have Docker installed before.

In your root folder add the following docker-compose.yml file:

# docker-compose.yaml

version: "3"

services:
  db:
    image: postgres
    restart: always
    ports:
      - "5432:5432"
    environment:
      POSTGRES_USER: rustproto
      POSTGRES_PASSWORD: rustproto

You can then run the command docker compose up -d to start your database (-d will start it in background mode).

2. Create the Protobuf definitions

Protobuf definitions allows 2 things:

Define how gRPC server and client communicate together
Generate SDKs using Protobuf generators

We will store them in a folder called proto

Creating the Auth service Protobuf definitions

We will need two methods in our service:

Register: To create a new user. It returns a Token.
Login: To generate a new Token if credentials are valid.

// proto/auth.proto

syntax = "proto3";

package auth;

message LoginRequest {
    string username = 1;
    string password = 2;
}

message RegisterRequest {
    string firstname = 1;
    string lastname = 2;
    string email = 3;
    string password = 4;
}

message Token {
    string access_token = 1;
}

service Auth {
    rpc Login (LoginRequest) returns (Token);
    rpc Register (RegisterRequest) returns (Token);
}

Creating the Greeting service definitions

To test that our authentication works we need a random service that will throw an unauthenticated status if the access token is invalid.

// proto/greeting.proto

syntax = "proto3";

package greeting;

message GreetRequest {
    string message = 1;
}

message GreetResponse {
    string message = 1;
}

service Greeting {
    rpc Greet (GreetRequest) returns (GreetResponse);
}

3. Setup Buf to generate SDKs

To generate SDKs we have different solutions:

Use tonic-build directly from Rust.
Use the Protobuf CLI protoc and the plugin protoc-gen-tonic.
Use Buf CLI to manage the SDKs generation.

The first two solutions would fit perfectly but as we want to overengineer our authentication system and think of the not coming future we will use the Buf CLI.

So make sure to install it.

Create the module configuration

Buf CLI works with workspaces and modules to easily split our protobuf definitions (by APIs, microservices, etc).

Let's make our proto folder our lonely module by creating a buf.yaml file with the following line:

# proto/buf.yaml

version: v1

Create the workspace configuration

Now we need to configure our Buf workspace by creating a buf.work.yaml at the root of our project:

# buf.work.yaml

version: v1
directories:
  - proto # <- We define our module in the workspace

Create the generator configuration

We will use four generators:

protoc-gen-prost The core code generation plugin
protoc-gen-serde Canonical JSON serialization of protobuf types
protoc-gen-tonic gRPC service generation for the Tonic framework
protoc-gen-prost-crate Generates an include file and cargo manifest for turn-key crates

Let's install them all using the following command:

$ cargo install protoc-gen-prost protoc-gen-prost-serde protoc-gen-tonic protoc-gen-prost-crate

Then, create the buf.gen.yaml file to configure the SDKs generation:

# buf.gen.yaml

version: v1
plugins:
  - plugin: prost # Generates the core code
    out: gen/src
    opt:
      - bytes=.
      - compile_well_known_types
      - extern_path=.google.protobuf=::pbjson_types
      - file_descriptor_set
  - plugin: prost-serde # Generates code compatible with JSON serde
    out: gen/src
  - plugin: tonic # Generates the Tonic services
    out: gen/src
    opt:
      - compile_well_known_types
      - extern_path=.google.protobuf=::pbjson_types
  - plugin: prost-crate # Makes the gen folder a crate
    out: gen
    opt:
      - gen_crate=gen/Cargo.toml

We now need to create the Cargo.toml that will be used as a template to generate the new one.

For simplicity we will use the generated one as a template so it can be replaced.

# gen/Cargo.toml

[package]
name = "protos"
version = "0.1.0"
edition = "2021"

[features]
default = ["proto_full"]
# @@protoc_deletion_point(features)
# This section is automatically generated by protoc-gen-prost-crate.
# Changes in this area may be lost on regeneration.
# @@protoc_insertion_point(features)

[dependencies]
bytes = "1.1.0"
prost = "0.12"
pbjson = "0.6"
pbjson-types = "0.6"
serde = "1.0"
tonic = { version = "0.10", features = ["gzip"] }

We can now generate our crate using the command buf generate.

🎉 Our SDK is now ready!

4. Setup the Rust project

Now that we have our SDK we can start creating the core of our server.
First create a Cargo.toml file and add our generate crate to the dependencies aswell as all the requirements for this project.

[package]
name = "proto-auth-demo"
version = "0.1.0"
edition = "2021"

[dependencies]
diesel = { version = "2.1.3", features = ["postgres"] }
dotenvy = "0.15.7"
prost = "0.12.1"
protobuf = "3.3.0"
serde = "1.0.190"
tokio = { version = "1.33.0", features = ["full"] }
tonic = "0.10.2"
protos = { path = "./gen" }
bcrypt = "0.15.0"
jwt = "0.16.0"
hmac = "0.12.1"
sha2 = "0.10.8"

[workspace]
members = [
    "gen"
]

We will need two environment variables:

DATABASE_URL: To authenticate to our PostgreSQL database
APP_KEY: To encrypt user passwords

Store them in a .env file, we will use them later using the dotenvy crate.

# .env

DATABASE_URL=postgres://rustproto:rustproto@localhost/rustproto
APP_KEY="9E3CnfSfsi9BGfX3Dea#tkbs#nDj&6d#6Y&jhNa!"

And we can now create our main.rs file:

// src/main.rs

use dotenvy::dotenv;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    dotenv().ok();
    Ok(())
}

And build our project using cargo build.

5. Use Diesel to manage our users

Setup Diesel

First install the Diesel CLI using the following command:

$ cargo install diesel_cli

You may need to install libpq-dev, libmysqlclient-dev and libsqlite3-dev to install the CLI.

Create the migrations folder:

$ diesel setup

And add the following lines to the top of your main.rs (we will create those modules later):

// src/main.rs

mod models;
mod schema;

Create the migration

Let's first generate the migration to create and drop the users table.

$ diesel migration generate create_users

-- migrations/*-create_users/up.sql
CREATE TABLE users (
    id SERIAL PRIMARY KEY,
    email VARCHAR(255) UNIQUE NOT NULL,
    password VARCHAR(255) NOT NULL,
    firstname VARCHAR(255) NOT NULL,
    lastname VARCHAR(255) NOT NULL
);

-- migrations/*-create_users/down.sql
DROP TABLE users;

And run the following command to run the migrations and generate the schema.rs file:

$ diesel migration run

Create the User model

Create a new file src/models.rs and add our User model to it and implements the methods to create a new User and find one by email.

// src/models.rs

use diesel::{
    ExpressionMethods, Insertable, PgConnection, QueryDsl, Queryable, RunQueryDsl, Selectable,
    SelectableHelper,
};

use crate::schema::users;

#[derive(Queryable, Selectable)]
#[diesel(table_name = users)]
#[diesel(check_for_backend(diesel::pg::Pg))]
pub struct User {
    pub id: i32,
    pub firstname: String,
    pub lastname: String,
    pub email: String,
    pub password: String,
}

#[derive(Insertable)]
#[diesel(table_name = users)]
pub struct NewUser<'a> {
    pub firstname: &'a str,
    pub lastname: &'a str,
    pub email: &'a str,
    pub password: &'a str,
}

impl User {
    pub fn create(
        conn: &mut PgConnection,
        firstname: &str,
        lastname: &str,
        email: &str,
        password: &str,
    ) -> Result<User, diesel::result::Error> {
        let new_user = NewUser {
            firstname,
            lastname,
            email,
            password,
        };

        diesel::insert_into(users::table)
            .values(new_user)
            .returning(User::as_returning())
            .get_result(conn)
    }

    pub fn find_by_email(conn: &mut PgConnection, email: &str) -> Option<User> {
        users::dsl::users
            .select(User::as_select())
            .filter(users::dsl::email.eq(email))
            .first(conn)
            .ok()
    }
}

Create the database connection

We now have to setup the database connection and we will be done with Diesel.

// src/main.rs

use std::env;

use diesel::{PgConnection, Connection};
use dotenvy::dotenv;

mod models;
mod schema;

pub fn connect_db() -> PgConnection {
    let database_url = env::var("DATABASE_URL").expect("DATABASE_URL must be set");
    PgConnection::establish(&database_url)
        .unwrap_or_else(|_| panic!("Error connecting to {}", database_url))
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    dotenv().ok();

    let mut database = connect_db();

    Ok(())
}

You can try it by creating a user using the database connection User.create(&mut database, "", "", "", "")

6. Implement the Tokio Services

It is now time to create the business logic for our authentication system.
We will start with the biggest part, the authentication.

Create the skeleton

use std::sync::{Arc, Mutex};

use diesel::PgConnection;
use protos::auth::{auth_server::Auth, LoginRequest, Token, RegisterRequest};
use tonic::{Request, Response, Status};

use crate::models::User;

pub struct Service {
    database: Arc<Mutex<PgConnection>>,
}

impl Service {
    pub fn new(database: PgConnection) -> Self {
        Self {
            database: Arc::new(Mutex::new(database))
        }
    }

    fn generate_token(user: User) -> Token {
        unimplemented!();
    }
}

#[tonic::async_trait]
impl Auth for Service {
    async fn login(&self, request: Request<LoginRequest>) -> Result<Response<Token>, Status> {
        unimplemented!();
    }

    async fn register(&self, request: Request<RegisterRequest>) -> Result<Response<Token>, Status> {
        unimplemented!();
    }
}

Start the Tonic Server and add our unimplemented service

In the main.rs file we are going to create a server, add our service to it and start it.

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    dotenv().ok();

    let database = connect_db();
    let addr = "[::1]:50051".parse()?;

    Server::builder()
        .add_service(AuthServer::new(auth::Service::new(database)))
        .serve(addr)
        .await?;

    Ok(())
}

You can try your server by using the following command to call the auth.Auth/Login function (package.Service/Method).

$ grpcurl -plaintext -import-path ./proto -proto auth.proto '[::1]:50051' auth.Auth/Login

You should see in the console of your server the following error:

thread 'tokio-runtime-worker' panicked at 'not implemented', src/auth.rs:22:9

This error has been thrown by the unimplemented! macro of our Auth Service, it works!

Generate an access token

We are going to use the jwt crate to create a JWT token. It will contains the following claims:

sub: The Subject of the JWT (our user id)
iat: Time at which the JWT was issued
exp: Time after which the JWT expires

Implement the register method

The register method is fairly simple, it directly takes the request message to create the entry in our database after encrypting the password.

// src/auth.rs

async fn register(&self, request: Request<RegisterRequest>) -> Result<Response<Token>, Status> {
    let database = self.database.lock();
    let data = request.into_inner();

    let password = bcrypt::hash(&data.password, 10)
        .map_err(|_| Status::unknown("Error while creating the user"))?;

    let user = NewUser {
        firstname: &data.firstname,
        lastname: &data.lastname,
        email: &data.email,
        password: &password,
    };

    let user = User::create(&mut database.unwrap(), user);

    unimplemented!();
}

You can already try it! It will throw an error because of the unimplemented but your user should be created in the database.

Implement the generate token method

We are going to split our function in two, the part for generating the claims and the one for encoding the token.

We are going to use the APP_KEY defined in our .env file to sign the JWT.

// src/auth.rs
pub struct GenerateTokenError;

pub struct GenerateClaimsError;

fn generate_claims(user: User) -> Result<BTreeMap<&'static str, String>, GenerateClaimsError> {
    let mut claims: BTreeMap<&str, String> = BTreeMap::new();

    claims.insert("sub", user.id.to_string());

    let current_timestamp = SystemTime::now()
        .duration_since(UNIX_EPOCH)
        .map_err(|_| GenerateClaimsError)?
        .as_secs();

    claims.insert("iat", current_timestamp.to_string());
    claims.insert("exp", String::from("3600"));

    Ok(claims)
}

fn generate_token(user: User) -> Result<Token, GenerateTokenError> {
    let app_key: String = env::var("APP_KEY").expect("env APP_KEY is not defined");

    let key: Hmac<Sha256> =
        Hmac::new_from_slice(app_key.as_bytes()).map_err(|_| GenerateTokenError)?;

    let claims = generate_claims(user).map_err(|_| GenerateTokenError)?;

    let access_token = claims.sign_with_key(&key).map_err(|_| GenerateTokenError)?;

    Ok(Token {
        access_token: access_token,
    })
}

And now, let's use this function to return an access token when a user is registered:

// src/auth.Rs
impl Auth for Service {
    [...]
    async fn register(&self, request: Request<RegisterRequest>) -> Result<Response<Token>, Status> {
        let database = self.database.lock();
        let data = request.into_inner();

        let password = bcrypt::hash(&data.password, 10)
            .map_err(|_| Status::unknown("Error while creating the user"))?;

        let user = NewUser {
            firstname: &data.firstname,
            lastname: &data.lastname,
            email: &data.email,
            password: &password,
        };

        let user = User::create(&mut database.unwrap(), user)
            .map_err(|_| Status::already_exists("User already exists in the database"))?;

        let token = generate_token(user).map_err(|_| Status::unknown("Cannot generate a token for the User"))?;

        Ok(Response::new(token))
    }
}

You can create a new user with the following command:

$ grpcurl -plaintext -import-path ./proto -proto auth.proto -d '{"firstname": "John", "lastname": "Doe", "email": "john@doe.com", "password": "rustproto"}' '[::1]:50051' auth.Auth/Register

And if you run it again, you will have the error AlreadyExists. 🎉

Implement the login method

The login method is now really simple, we try to find a user correspond to the email in the message, verify if the password is correct and use the generate_token method to return the Response.

// src/auth.rs
impl Auth for Service {
    [...]
    async fn login(&self, request: Request<LoginRequest>) -> Result<Response<Token>, Status> {
        let data = request.into_inner();

        let database = self.database.lock();

        let user = User::find_by_email(&mut database.unwrap(), &data.email)
            .ok_or(Status::unauthenticated("Invalid email or password"))?;

        match verify(data.password, &user.password) {
            Ok(true) => (),
            Ok(false) | Err(_) => return Err(Status::unauthenticated("Invalid email or password")),
        };

        let reply = generate_token(user)
            .map_err(|_| Status::unauthenticated("Invalid email or password"))?;

        Ok(Response::new(reply))
    }
    [...]
}

And we can now generate a token using the email and password of our previously registered user!

$ grpcurl -plaintext -import-path ./proto -proto auth.proto -d '{"email": "john@doe.com", "password": "rustproto"}' '[::1]:50051' auth.Auth/Login

Implement the verify token method

As this post already starts to be really long, we will make the verification simple, we will only check for the signature and return true if it is valid.

// src/auth.rs
pub struct VerifyTokenError;

pub fn verify_token(token: &str) -> Result<bool, VerifyTokenError> {
    let app_key: String = env::var("APP_KEY").expect("env APP_KEY is not defined");

    let key: Hmac<Sha256> =
        Hmac::new_from_slice(app_key.as_bytes()).map_err(|_| VerifyTokenError)?;

    Ok(token
        .verify_with_key(&key)
        .map(|_: HashMap<String, String>| true)
        .unwrap_or(false))
}

Implement the Greeting service

In a new src/greeting.rs file we are going to implement our Greeting service using the trait provided by our generated SDk.

We will verify that we have a x-authorization token in the metadata and verify its value.

// src/greeting.rs

use protos::greeting::{greeting_server::Greeting, GreetRequest, GreetResponse};
use tonic::{Request, Response, Status};

use crate::auth;

#[derive(Default)]
pub struct Service {}

#[tonic::async_trait]
impl Greeting for Service {
    async fn greet(
        &self,
        request: Request<GreetRequest>,
    ) -> Result<Response<GreetResponse>, Status> {
        let token = request
            .metadata()
            .get("x-authorization")
            .ok_or(Status::unauthenticated("No access token specified"))?
            .to_str()
            .map_err(|_| Status::unauthenticated("No access token specified"))?;

        match auth::verify_token(token) {
            Ok(true) => (),
            Err(_) | Ok(false) => return Err(Status::unauthenticated("Invalid token")),
        }

        let data = request.into_inner();

        Ok(Response::new(GreetResponse {
            message: format!("{} {}", data.message, "Pong!"),
        }))
    }
}

We can now add our service to the Tonic server in main.rs.

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    dotenv().ok();

    let database = connect_db();
    let addr = "[::1]:50051".parse()?;

    Server::builder()
        .add_service(AuthServer::new(auth::Service::new(database)))
        .add_service(GreetingServer::new(greeting::Service::default()))
        .serve(addr)
        .await?;

    Ok(())
}

It's now time to try our authentication system! You can use the following command and replace the access token with one generated with the Login or Register method:

$ grpcurl -plaintext -import-path ./proto -proto greeting.proto -H 'x-authorization: <access_token>' -d '{"message": "Ping!" }' '[::1]:50051' greeting.Greeting/Greet

Try to put a wrong access token, you will get a unauthenticated status!

You made it 🚀

Rust and Protobuf are new to me, but it is exactly the kind of project that helps me having a great learning experience on things that have a steep learning curve and I hope that it also helped you having a better understanding of Rust, Protobuf and gRPC.

You can find the repository here https://github.com/kerwanp/rust-proto-demo

In the next blog post we will learn how to consume this API in NextJS, so make sure to stay tuned by following me on dev.to/martinp and on Twitter @PaucotMartin.

And if you are stuck, have any question or recommendations, I am here in the comments section!

Roll your own auth with Rust and Protobuf

Martin PAUCOT — Sat, 28 Oct 2023 15:58:33 +0000

Introduction

Before starting, take a deep breath, maybe 5 minutes of meditation and prepare a drink because this tutorial imply Rust code. Not complex, but Rust code.

But first, let's talk about Protobuf, what is it?

1. What is Protobuf?

According to the documentation:

"Protocol Buffers are language-neutral, platform-neutral extensible mechanisms for serializing structured data."

JSON is really flexible when you want to share data through services you can decode JSON without knowing it's structure first.
But it is unstructured, takes a lot of space and bandwith.

With Protocol Buffers you define a message and its structure that must be known by both the server and the client to encode and decode.

// user.proto

syntax = "proto3";

message User {
    string firstname = 1;
    string lastname = 2;
    string email = 3;
}

You can then use generators to create the SDKs for your favorites languages. You can generate a Javascript one for your frontend and a Rust one for your backend.

// user.proto

syntax = "proto3";

message LoginRequest {
    string email = 1;
    string password = 2;
}

service Auth {
    rpc Login (LoginRequest) returns (User);
}

2. What are you we to accomplish

Create a PostgreSQL database using Docker and Docker compose
Create Protobuf definitions and use Buf to generate the SDKs
Setup a gRPC server in Rust using Tonic
Create a JWT authentication system with Diesel and Tonic

3. Requirements

You must know how JWT works as I will not explain it in this blog post.

You must have all the required tools installed:

If you are lost somewhere, or you directly want to go straigth to the code, the repository is available here https://github.com/kerwanp/rust-proto-demo.

Get started

1. Create the PostgreSQL database

As we want to persist our users we need a database. We are going to use PostgreSQL.

For simplicity of use, we will use Docker Compose to manage it. Make sure you have Docker installed before.

In your root folder add the following docker-compose.yml file:

# docker-compose.yaml

version: "3"

services:
  db:
    image: postgres
    restart: always
    ports:
      - "5432:5432"
    environment:
      POSTGRES_USER: rustproto
      POSTGRES_PASSWORD: rustproto

You can then run the command docker compose up -d to start your database (-d will start it in background mode).

2. Create the Protobuf definitions

Protobuf definitions allows 2 things:

Define how gRPC server and client communicate together
Generate SDKs using Protobuf generators

We will store them in a folder called proto

Creating the Auth service Protobuf definitions

We will need two methods in our service:

Register: To create a new user. It returns a Token.
Login: To generate a new Token if credentials are valid.

// proto/auth.proto

syntax = "proto3";

package auth;

message LoginRequest {
    string username = 1;
    string password = 2;
}

message RegisterRequest {
    string firstname = 1;
    string lastname = 2;
    string email = 3;
    string password = 4;
}

message Token {
    string access_token = 1;
}

service Auth {
    rpc Login (LoginRequest) returns (Token);
    rpc Register (RegisterRequest) returns (Token);
}

Creating the Greeting service definitions

To test that our authentication works we need a random service that will throw an unauthenticated status if the access token is invalid.

// proto/greeting.proto

syntax = "proto3";

package greeting;

message GreetRequest {
    string message = 1;
}

message GreetResponse {
    string message = 1;
}

service Greeting {
    rpc Greet (GreetRequest) returns (GreetResponse);
}

3. Setup Buf to generate SDKs

To generate SDKs we have different solutions:

Use tonic-build directly from Rust.
Use the Protobuf CLI protoc and the plugin protoc-gen-tonic.
Use Buf CLI to manage the SDKs generation.

The first two solutions would fit perfectly but as we want to overengineer our authentication system and think of the not coming future we will use the Buf CLI.

So make sure to install it.

Create the module configuration

Buf CLI works with workspaces and modules to easily split our protobuf definitions (by APIs, microservices, etc).

Let's make our proto folder our lonely module by creating a buf.yaml file with the following line:

# proto/buf.yaml

version: v1

Create the workspace configuration

Now we need to configure our Buf workspace by creating a buf.work.yaml at the root of our project:

# buf.work.yaml

version: v1
directories:
  - proto # <- We define our module in the workspace

Create the generator configuration

We will use four generators:

protoc-gen-prost The core code generation plugin
protoc-gen-serde Canonical JSON serialization of protobuf types
protoc-gen-tonic gRPC service generation for the Tonic framework
protoc-gen-prost-crate Generates an include file and cargo manifest for turn-key crates

Let's install them all using the following command:

$ cargo install protoc-gen-prost protoc-gen-prost-serde protoc-gen-tonic protoc-gen-prost-crate

Then, create the buf.gen.yaml file to configure the SDKs generation:

# buf.gen.yaml

version: v1
plugins:
  - plugin: prost # Generates the core code
    out: gen/src
    opt:
      - bytes=.
      - compile_well_known_types
      - extern_path=.google.protobuf=::pbjson_types
      - file_descriptor_set
  - plugin: prost-serde # Generates code compatible with JSON serde
    out: gen/src
  - plugin: tonic # Generates the Tonic services
    out: gen/src
    opt:
      - compile_well_known_types
      - extern_path=.google.protobuf=::pbjson_types
  - plugin: prost-crate # Makes the gen folder a crate
    out: gen
    opt:
      - gen_crate=gen/Cargo.toml

We now need to create the Cargo.toml that will be used as a template to generate the new one.

For simplicity we will use the generated one as a template so it can be replaced.

# gen/Cargo.toml

[package]
name = "protos"
version = "0.1.0"
edition = "2021"

[features]
default = ["proto_full"]
# @@protoc_deletion_point(features)
# This section is automatically generated by protoc-gen-prost-crate.
# Changes in this area may be lost on regeneration.
# @@protoc_insertion_point(features)

[dependencies]
bytes = "1.1.0"
prost = "0.12"
pbjson = "0.6"
pbjson-types = "0.6"
serde = "1.0"
tonic = { version = "0.10", features = ["gzip"] }

We can now generate our crate using the command buf generate.

🎉 Our SDK is now ready!

4. Setup the Rust project

Now that we have our SDK we can start creating the core of our server.
First create a Cargo.toml file and add our generate crate to the dependencies aswell as all the requirements for this project.

[package]
name = "proto-auth-demo"
version = "0.1.0"
edition = "2021"

[dependencies]
diesel = { version = "2.1.3", features = ["postgres"] }
dotenvy = "0.15.7"
prost = "0.12.1"
protobuf = "3.3.0"
serde = "1.0.190"
tokio = { version = "1.33.0", features = ["full"] }
tonic = "0.10.2"
protos = { path = "./gen" }
bcrypt = "0.15.0"
jwt = "0.16.0"
hmac = "0.12.1"
sha2 = "0.10.8"

[workspace]
members = [
    "gen"
]

We will need two environment variables:

DATABASE_URL: To authenticate to our PostgreSQL database
APP_KEY: To encrypt user passwords

Store them in a .env file, we will use them later using the dotenvy crate.

# .env

DATABASE_URL=postgres://rustproto:rustproto@localhost/rustproto
APP_KEY="9E3CnfSfsi9BGfX3Dea#tkbs#nDj&6d#6Y&jhNa!"

And we can now create our main.rs file:

// src/main.rs

use dotenvy::dotenv;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    dotenv().ok();
    Ok(())
}

And build our project using cargo build.

5. Use Diesel to manage our users

Setup Diesel

First install the Diesel CLI using the following command:

$ cargo install diesel_cli

You may need to install libpq-dev, libmysqlclient-dev and libsqlite3-dev to install the CLI.

Create the migrations folder:

$ diesel setup

And add the following lines to the top of your main.rs (we will create those modules later):

// src/main.rs

mod models;
mod schema;

Create the migration

Let's first generate the migration to create and drop the users table.

$ diesel migration generate create_users

-- migrations/*-create_users/up.sql
CREATE TABLE users (
    id SERIAL PRIMARY KEY,
    email VARCHAR(255) UNIQUE NOT NULL,
    password VARCHAR(255) NOT NULL,
    firstname VARCHAR(255) NOT NULL,
    lastname VARCHAR(255) NOT NULL
);

-- migrations/*-create_users/down.sql
DROP TABLE users;

And run the following command to run the migrations and generate the schema.rs file:

$ diesel migration run

Create the User model

Create a new file src/models.rs and add our User model to it and implements the methods to create a new User and find one by email.

// src/models.rs

use diesel::{
    ExpressionMethods, Insertable, PgConnection, QueryDsl, Queryable, RunQueryDsl, Selectable,
    SelectableHelper,
};

use crate::schema::users;

#[derive(Queryable, Selectable)]
#[diesel(table_name = users)]
#[diesel(check_for_backend(diesel::pg::Pg))]
pub struct User {
    pub id: i32,
    pub firstname: String,
    pub lastname: String,
    pub email: String,
    pub password: String,
}

#[derive(Insertable)]
#[diesel(table_name = users)]
pub struct NewUser<'a> {
    pub firstname: &'a str,
    pub lastname: &'a str,
    pub email: &'a str,
    pub password: &'a str,
}

impl User {
    pub fn create(
        conn: &mut PgConnection,
        firstname: &str,
        lastname: &str,
        email: &str,
        password: &str,
    ) -> Result<User, diesel::result::Error> {
        let new_user = NewUser {
            firstname,
            lastname,
            email,
            password,
        };

        diesel::insert_into(users::table)
            .values(new_user)
            .returning(User::as_returning())
            .get_result(conn)
    }

    pub fn find_by_email(conn: &mut PgConnection, email: &str) -> Option<User> {
        users::dsl::users
            .select(User::as_select())
            .filter(users::dsl::email.eq(email))
            .first(conn)
            .ok()
    }
}

Create the database connection

We now have to setup the database connection and we will be done with Diesel.

// src/main.rs

use std::env;

use diesel::{PgConnection, Connection};
use dotenvy::dotenv;

mod models;
mod schema;

pub fn connect_db() -> PgConnection {
    let database_url = env::var("DATABASE_URL").expect("DATABASE_URL must be set");
    PgConnection::establish(&database_url)
        .unwrap_or_else(|_| panic!("Error connecting to {}", database_url))
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    dotenv().ok();

    let mut database = connect_db();

    Ok(())
}

You can try it by creating a user using the database connection User.create(&mut database, "", "", "", "")

5. Implement the Tokio Services

It is now time to create the business logic for our authentication system.
We will start with the biggest part, the authentication.

Create the skeleton

use std::sync::{Arc, Mutex};

use diesel::PgConnection;
use protos::auth::{auth_server::Auth, LoginRequest, Token, RegisterRequest};
use tonic::{Request, Response, Status};

use crate::models::User;

pub struct Service {
    database: Arc<Mutex<PgConnection>>,
}

impl Service {
    pub fn new(database: PgConnection) -> Self {
        Self {
            database: Arc::new(Mutex::new(database))
        }
    }

    fn generate_token(user: User) -> Token {
        unimplemented!();
    }
}

#[tonic::async_trait]
impl Auth for Service {
    async fn login(&self, request: Request<LoginRequest>) -> Result<Response<Token>, Status> {
        unimplemented!();
    }

    async fn register(&self, request: Request<RegisterRequest>) -> Result<Response<Token>, Status> {
        unimplemented!();
    }
}

Start the Tonic Server and add our unimplemented service

In the main.rs file we are going to create a server, add our service to it and start it.

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    dotenv().ok();

    let database = connect_db();
    let addr = "[::1]:50051".parse()?;

    Server::builder()
        .add_service(AuthServer::new(auth::Service::new(database)))
        .serve(addr)
        .await?;

    Ok(())
}

You can try your server by using the following command to call the auth.Auth/Login function (package.Service/Method).

$ grpcurl -plaintext -import-path ./proto -proto auth.proto '[::1]:50051' auth.Auth/Login

You should see in the console of your server the following error:

thread 'tokio-runtime-worker' panicked at 'not implemented', src/auth.rs:22:9

This error has been thrown by the unimplemented! macro of our Auth Service, it works!

Generate an access token

We are going to use the jwt crate to create a JWT token. It will contains the following claims:

sub: The Subject of the JWT (our user id)
iat: Time at which the JWT was issued
exp: Time after which the JWT expires

Implement the register method

The register method is fairly simple, it directly takes the request message to create the entry in our database after encrypting the password.

// src/auth.rs

async fn register(&self, request: Request<RegisterRequest>) -> Result<Response<Token>, Status> {
    let database = self.database.lock();
    let data = request.into_inner();

    let password = bcrypt::hash(&data.password, 10)
        .map_err(|_| Status::unknown("Error while creating the user"))?;

    let user = NewUser {
        firstname: &data.firstname,
        lastname: &data.lastname,
        email: &data.email,
        password: &password,
    };

    let user = User::create(&mut database.unwrap(), user);

    unimplemented!();
}

You can already try it! It will throw an error because of the unimplemented but your user should be created in the database.

Implement the generate token method

We are going to split our function in two, the part for generating the claims and the one for encoding the token.

We are going to use the APP_KEY defined in our .env file to sign the JWT.

// src/auth.rs
pub struct GenerateTokenError;

pub struct GenerateClaimsError;

fn generate_claims(user: User) -> Result<BTreeMap<&'static str, String>, GenerateClaimsError> {
    let mut claims: BTreeMap<&str, String> = BTreeMap::new();

    claims.insert("sub", user.id.to_string());

    let current_timestamp = SystemTime::now()
        .duration_since(UNIX_EPOCH)
        .map_err(|_| GenerateClaimsError)?
        .as_secs();

    claims.insert("iat", current_timestamp.to_string());
    claims.insert("exp", String::from("3600"));

    Ok(claims)
}

fn generate_token(user: User) -> Result<Token, GenerateTokenError> {
    let app_key: String = env::var("APP_KEY").expect("env APP_KEY is not defined");

    let key: Hmac<Sha256> =
        Hmac::new_from_slice(app_key.as_bytes()).map_err(|_| GenerateTokenError)?;

    let claims = generate_claims(user).map_err(|_| GenerateTokenError)?;

    let access_token = claims.sign_with_key(&key).map_err(|_| GenerateTokenError)?;

    Ok(Token {
        access_token: access_token,
    })
}

And now, let's use this function to return an access token when a user is registered:

// src/auth.Rs
impl Auth for Service {
    [...]
    async fn register(&self, request: Request<RegisterRequest>) -> Result<Response<Token>, Status> {
        let database = self.database.lock();
        let data = request.into_inner();

        let password = bcrypt::hash(&data.password, 10)
            .map_err(|_| Status::unknown("Error while creating the user"))?;

        let user = NewUser {
            firstname: &data.firstname,
            lastname: &data.lastname,
            email: &data.email,
            password: &password,
        };

        let user = User::create(&mut database.unwrap(), user)
            .map_err(|_| Status::already_exists("User already exists in the database"))?;

        let token = generate_token(user).map_err(|_| Status::unknown("Cannot generate a token for the User"))?;

        Ok(Response::new(token))
    }
}

You can create a new user with the following command:

$ grpcurl -plaintext -import-path ./proto -proto auth.proto -d '{"firstname": "John", "lastname": "Doe", "email": "john@doe.com", "password": "rustproto"}' '[::1]:50051' auth.Auth/Register

And if you run it again, you will have the error AlreadyExists. 🎉

Implement the login method

The login method is now really simple, we try to find a user correspond to the email in the message, verify if the password is correct and use the generate_token method to return the Response.

// src/auth.rs
impl Auth for Service {
    [...]
    async fn login(&self, request: Request<LoginRequest>) -> Result<Response<Token>, Status> {
        let data = request.into_inner();

        let database = self.database.lock();

        let user = User::find_by_email(&mut database.unwrap(), &data.email)
            .ok_or(Status::unauthenticated("Invalid email or password"))?;

        match verify(data.password, &user.password) {
            Ok(true) => (),
            Ok(false) | Err(_) => return Err(Status::unauthenticated("Invalid email or password")),
        };

        let reply = generate_token(user)
            .map_err(|_| Status::unauthenticated("Invalid email or password"))?;

        Ok(Response::new(reply))
    }
    [...]
}

And we can now generate a token using the email and password of our previously registered user!

$ grpcurl -plaintext -import-path ./proto -proto auth.proto -d '{"email": "john@doe.com", "password": "rustproto"}' '[::1]:50051' auth.Auth/Login

Implement the verify token method

As this post already starts to be really long, we will make the verification simple, we will only check for the signature and return true if it is valid.

// src/auth.rs
pub struct VerifyTokenError;

pub fn verify_token(token: &str) -> Result<bool, VerifyTokenError> {
    let app_key: String = env::var("APP_KEY").expect("env APP_KEY is not defined");

    let key: Hmac<Sha256> =
        Hmac::new_from_slice(app_key.as_bytes()).map_err(|_| VerifyTokenError)?;

    Ok(token
        .verify_with_key(&key)
        .map(|_: HashMap<String, String>| true)
        .unwrap_or(false))
}

Implement the Greeting service

In a new src/greeting.rs file we are going to implement our Greeting service using the trait provided by our generated SDk.

We will verify that we have a x-authorization token in the metadata and verify its value.

// src/greeting.rs

use protos::greeting::{greeting_server::Greeting, GreetRequest, GreetResponse};
use tonic::{Request, Response, Status};

use crate::auth;

#[derive(Default)]
pub struct Service {}

#[tonic::async_trait]
impl Greeting for Service {
    async fn greet(
        &self,
        request: Request<GreetRequest>,
    ) -> Result<Response<GreetResponse>, Status> {
        let token = request
            .metadata()
            .get("x-authorization")
            .ok_or(Status::unauthenticated("No access token specified"))?
            .to_str()
            .map_err(|_| Status::unauthenticated("No access token specified"))?;

        match auth::verify_token(token) {
            Ok(true) => (),
            Err(_) | Ok(false) => return Err(Status::unauthenticated("Invalid token")),
        }

        let data = request.into_inner();

        Ok(Response::new(GreetResponse {
            message: format!("{} {}", data.message, "Pong!"),
        }))
    }
}

We can now add our service to the Tonic server in main.rs.

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    dotenv().ok();

    let database = connect_db();
    let addr = "[::1]:50051".parse()?;

    Server::builder()
        .add_service(AuthServer::new(auth::Service::new(database)))
        .add_service(GreetingServer::new(greeting::Service::default()))
        .serve(addr)
        .await?;

    Ok(())
}

It's now time to try our authentication system! You can use the following command and replace the access token with one generated with the Login or Register method:

$ grpcurl -plaintext -import-path ./proto -proto greeting.proto -H 'x-authorization: <access_token>' -d '{"message": "Ping!" }' '[::1]:50051' greeting.Greeting/Greet

Try to put a wrong access token, you will get a unauthenticated status!

You made it 🚀

Rust and Protobuf are new to me, but it is exactly the kind of project that helps me having a great learning experience on things that have a steep learning curve.

You can find the repository here https://github.com/kerwanp/rust-proto-demo

In the next blog post we will learn how to consume this API in NextJS, so make sure to stay tuned by following me on [dev.to]

How to do stunning 3D with pure HTML/CSS

Martin PAUCOT — Fri, 08 Sep 2023 22:34:17 +0000

HTML and CSS may be the bedrock of 2D web design, but hidden within their virtual toolbox are the secrets to creating breathtaking 3D perspectives. Imagine crafting interactive interfaces that transcend flatland into a world where the X, Y, and Z axes dance together. In this blog post, we'll unveil the magic of HTML and CSS's 3D capabilities, guiding you through the essential properties and techniques to create stunning 3D visuals.

Like this one!

Introduction

Harnessing Perspective in HTML

Although HTML and CSS are primarily designed for constructing 2D layouts, there are still 3 axes (X, Y & Z) enabling the creation of perspective through specific CSS properties.

The `perspective` property

The perspective property alters the user's perspective by adjusting the position of the vanishing Point along the Z-axis.

For example, with the following code, utilizing the following code snippet will establish a distance of 800px between the user and the plane (Z-axis).



.perspective {
  position: relative;
  perspective: 800px;
}



<div class="perspective"></div>

The `perspective-origin` property

The perspective-origin property determines the user's position along the X and Y axes.

To view an object from above, you can employ the following code:



.perspective {
  position: relative;
  perspective: 800px;
  perspective-origin: 50% -200px;
}

In this example, we are centered on the X-axis (50%) and positioned -200px on the Y-axis.

Now that you have grasped the essentials of 3D in HTML and CSS, let's embark on creating our own 3D cube!

1. Building the Foundation

For our HTML structure, we will construct a perspective plane and add a div element for each face of our cube.



<div class="perspective">
  <div class="box">
    <div class="face top"></div>
    <div class="face bottom"></div>
    <div class="face back"></div>
    <div class="face front"></div>
    <div class="face left"></div>
    <div class="face right"></div>
  </div>
</div>

Now, let's imbue our perspective plane with a specific perspective. In our case, we desire an overhead view of our cube.



.perspective {
  perspective: 800px;
  perspective-origin: 50% -200px;
}

Then, we'll assign dimensions to our box and apply styling to our individual faces. To keep the 3D rendering within the perspective plane in our div, we incorporate transform-style: preserve-3d.



.box {  
  width: var(--size);
  aspect-ratio: 1;  

  position: relative;
  transform-style: preserve-3d;
}

.face {
  position: absolute;
  width: var(--size);
  aspect-ratio: 1;

  background-color: hsl(
    39,
    100%,
    66%
  );

  transform-style: preserve-3d;
}

At this point, you should see all the faces superimposed at the same location, forming a square.

2. Creating the Pattern

Now, let's fabricate the 2D pattern of our cube, which we will then fold like a piece of paper. Additionally, we will incorporate a rotation animation for a better 3D visualization.

Creating a pattern simplifies the process of rotating and positioning elements accurately, preventing confusion with inverted faces.




.box {  
  animation: rotate 4s linear infinite;
}

.face.front {
}

.face.bottom {
  top: 100%;
}

.face.top {
  bottom: 100%;
}

.face.back {
  bottom: 200%;
}

.face.right {
  left: 100%;
}

.face.left {
  right: 100%;
}

@keyframes rotate {
  from {
    transform: rotate(0);
  }
  to {
    transform: rotateY(360deg);
  }
}

Now, you should see a cross-shaped configuration:

3. Folding the Faces to Craft the Cube

Our next step involves folding the faces to create the cube effect. This entails utilizing rotateX(), rotateY() and specifying the vertex from which the folding originates via transform-origin.

For the back face we need to adjust the transform-origin both the Y and Z axes since it is located two faces away from the front face.



.face.front {
}

.face.bottom {
  top: 100%;
  transform-origin: top;
  transform: rotateX(-90deg);
}

.face.top {
  bottom: 100%;
  transform-origin: bottom;
  transform: rotateX(90deg);
}

.face.back {
  bottom: 200%;
  transform-origin: center 150px -50px;
  transform: rotateX(180deg);
}

.face.right {
  left: 100%;
  transform-origin: left;
  transform: rotateY(90deg);
}

.face.left {
  right: 100%;
  transform-origin: right;
  transform: rotateY(-90deg);
}

4. Applying Shading to the Faces

To achieve a convincing 3D effect, we need to introduce shading to the faces. Although CSS lacks native lighting, we can create the illusion of depth by varying the lightness of colors.

We'll utilize the lightness property of hsl() to produce distinct shades for each face.



.face {  
  background-color: hsl(
    39,
    100%,
    var(--lightness)
  );
}

.face.front {
  --lightness: 66%;
}

.face.bottom {
  --lightness: 74%;
}

.face.top {
  --lightness: 74%;
}

.face.back {
  --lightness: 78%;
}

.face.right {
  --lightness: 70%;
}

.face.left {
  --lightness: 70%;
}

Now, you should behold a splendid 3D cube, crafted solely with HTML and CSS!

I hope this tutorial has ignited your imagination for creating captivating 3D animations with pure HTML and CSS.

Feel free to share your creations in the comments section!
And for more content like this, be sure to follow me @PaucotMartin on X (formerly Twitter).

How to use the Pulumi Operator to build Docker Images

Martin PAUCOT — Sun, 21 May 2023 11:42:33 +0000

I recently faced a problem when trying to build a Docker image using the @pulumi/docker provider.

In this Blog Post I explain my journey on how I use the Pulumi Operator to build Docker Images and what are the problems I encountered.

If you simply want the solution, click here to scroll down

import * as docker from '@pulumi/docker'
import * as kubernetes from '@pulumi/kubernetes'

const image = new docker.Image("app", {
    build: {
        context: "../",
        args: {
            BUILDKIT_INLINE_CACHE: "1",
        },
        cacheFrom: {
            images: ["ghcr.io/kerwanp/pulumi-preview-test:latest"]
        }
    },
    imageName: "ghcr.io/kerwanp/pulumi-preview-test:latest",
})

const appLabels = { app: "preview-test" }

const deployment = new kubernetes.apps.v1.Deployment("app", {
    spec: {
        replicas: 1,
        selector: { matchLabels: appLabels },
        template: {
            metadata: { labels: appLabels },
            spec: {
                imagePullSecrets: [
                    {
                        name: 'regcred'
                    }
                ],
                containers: [
                    {
                        name: "app",
                        image: image.imageName,
                        imagePullPolicy: 'Always',
                    }
                ]
            }
        }
    }
})

And then deploy it by creating a stack using the Pulumi Operator:

---
apiVersion: pulumi.com/v1
kind: Stack
metadata:
  name: app-stack
  namespace: pulumi
spec:
  envRefs:
    PULUMI_ACCESS_TOKEN:
      type: Secret
      secret:
        name: pulumi-secret
        key: pulumiAccessToken
  gitAuth:
    basicAuth:
      password:
        type: Secret
        secret:
          name: pulumi-secret
          key: githubToken
      userName:
        type: Literal
        literal:
          value: kerwanp
  stack: kerwanp/preview-test/test
  projectRepo: https://github.com/kerwanp/pulumi-preview-test
  repoDir: deploy
  branch: "refs/heads/main"
  destroyOnFinalize: true

The problem

When the Pulumi Operator tries to deploy the stack it throws the following error:

failed to run update: exit status 255code: 255
stdout: Updating (test)

View Live: https://app.pulumi.com/kerwanp/preview-test/test/updates/1


 +  pulumi:pulumi:Stack preview-test-test creating (0s) 
@ Updating......
 +  pulumi:pulumi:Stack preview-test-test creating (2s) panic: runtime error: invalid memory address or nil pointer dereference
 +  pulumi:pulumi:Stack preview-test-test creating (2s) [signal SIGSEGV: segmentation violation code=0x1 addr=0x29 pc=0x124befc]
 +  pulumi:pulumi:Stack preview-test-test creating (2s) goroutine 27 [running]:
 +  pulumi:pulumi:Stack preview-test-test creating (2s) github.com/pulumi/pulumi-docker/provider/v4.dockerHybridProvider.Configure({{}, {0x21b7ca0, 0x26157, 0x26157}, {0x17dd3fc, 0x6}, {0x17fe470, 0xc0000d1400}, {0x17fe3c0, 0xc0000b0690}}, ...)
 +  pulumi:pulumi:Stack preview-test-test creating (2s)     /home/runner/work/pulumi-docker/pulumi-docker/provider/hybrid.go:93 +0x17c
 +  pulumi:pulumi:Stack preview-test-test creating (2s) github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceProvider_Configure_Handler.func1({0x17f1f08, 0xc00042e6f0}, {0x14c4e00?, 0xc0007d7900})
 +  pulumi:pulumi:Stack preview-test-test creating (2s)     /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.64.0/proto/go/provider_grpc.pb.go:462 +0x78
 +  pulumi:pulumi:Stack preview-test-test creating (2s) github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1({0x17f1f08, 0xc0006ffe90}, {0x14c4e00, 0xc0007d7900}, 0xc000136ac0, 0xc00043eca8)
 +  pulumi:pulumi:Stack preview-test-test creating (2s)     /home/runner/go/pkg/mod/github.com/grpc-ecosystem/grpc-opentracing@v0.0.0-20180507213350-8e809c8a8645/go/otgrpc/server.go:57 +0x3e8
 +  pulumi:pulumi:Stack preview-test-test creating (2s) github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceProvider_Configure_Handler({0x153bb40?, 0xc0007644b0}, {0x17f1f08, 0xc0006ffe90}, 0xc0005da8c0, 0xc000653960)
 +  pulumi:pulumi:Stack preview-test-test creating (2s)     /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.64.0/proto/go/provider_grpc.pb.go:464 +0x138
 +  pulumi:pulumi:Stack preview-test-test creating (2s) google.golang.org/grpc.(*Server).processUnaryRPC(0xc000234000, {0x17fa100, 0xc000702b60}, 0xc0003c5e60, 0xc00075cba0, 0x21f98e8, 0x0)
 +  pulumi:pulumi:Stack preview-test-test creating (2s)     /home/runner/go/pkg/mod/google.golang.org/grpc@v1.54.0/server.go:1345 +0xdf3
 +  pulumi:pulumi:Stack preview-test-test creating (2s) google.golang.org/grpc.(*Server).handleStream(0xc000234000, {0x17fa100, 0xc000702b60}, 0xc0003c5e60, 0x0)
 +  pulumi:pulumi:Stack preview-test-test creating (2s)     /home/runner/go/pkg/mod/google.golang.org/grpc@v1.54.0/server.go:1722 +0xa36
 +  pulumi:pulumi:Stack preview-test-test creating (2s) google.golang.org/grpc.(*Server).serveStreams.func1.2()
 +  pulumi:pulumi:Stack preview-test-test creating (2s)     /home/runner/go/pkg/mod/google.golang.org/grpc@v1.54.0/server.go:966 +0x98
 +  pulumi:pulumi:Stack preview-test-test creating (2s) created by google.golang.org/grpc.(*Server).serveStreams.func1
 +  pulumi:pulumi:Stack preview-test-test creating (2s)     /home/runner/go/pkg/mod/google.golang.org/grpc@v1.54.0/server.go:964 +0x28a
    docker:index:Image app  error: error reading from server: EOF
    docker:index:Image app **failed** 1 error
 +  pulumi:pulumi:Stack preview-test-test created (2s) 19 messages

Diagnostics:
  docker:index:Image (app):
    error: error reading from server: EOF

  pulumi:pulumi:Stack (preview-test-test):
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x29 pc=0x124befc]
    goroutine 27 [running]:
    github.com/pulumi/pulumi-docker/provider/v4.dockerHybridProvider.Configure({{}, {0x21b7ca0, 0x26157, 0x26157}, {0x17dd3fc, 0x6}, {0x17fe470, 0xc0000d1400}, {0x17fe3c0, 0xc0000b0690}}, ...)
        /home/runner/work/pulumi-docker/pulumi-docker/provider/hybrid.go:93 +0x17c
    github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceProvide

r_Configure_Handler.func1({0x17f1f08, 0xc00042e6f0}, {0x14c4e00?, 0xc0007d7900})
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.64.0/proto/go/provider_grpc.pb.go:462 +0x78
    github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1({0x17f1f08, 0xc0006ffe90}, {0x14c4e00, 0xc0007d7900}, 0xc000136ac0, 0xc00043eca8)
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/grpc-opentracing@v0.0.0-20180507213350-8e809c8a8645/go/otgrpc/server.go:57 +0x3e8
    github.com/pulumi/pulumi/sdk/v3/proto/go._ResourceProvider_Configure_Handler({0x153bb40?, 0xc0007644b0}, {0x17f1f08, 0xc0006ffe90}, 0xc0005da8c0, 0xc000653960)
        /home/runner/go/pkg/mod/github.com/pulumi/pulumi/sdk/v3@v3.64.0/proto/go/provider_grpc.pb.go:464 +0x138
    google.golang.org/grpc.(*Server).processUnaryRPC(0xc000234000, {0x17fa100, 0xc000702b60}, 0xc0003c5e60, 0xc00075cba0, 0x21f98e8, 0x0)
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.54.0/server.go:1345 +0xdf3
    google.golang.org/grpc.(*Server).handleStream(0xc000234000, {0x17fa100, 0xc000702b60}, 0xc0003c5e60, 0x0)
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.54.0/server.go:1722 +0xa36
    google.golang.org/grpc.(*Server).serveStreams.func1.2()
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.54.0/server.go:966 +0x98
    created by google.golang.org/grpc.(*Server).serveStreams.func1
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.54.0/server.go:964 +0x28a

Resources:
    + 1 created

Duration: 4s


stderr: warning: A new version of Pulumi is available. To upgrade from version '3.66.0' to '3.68.0', visit https://pulumi.com/docs/reference/install/ for manual instructions and release notes.

The Debuging Process

Replicating the issue

I opened a shell on the Pulumi Operator Pod to see if I can replicate the pulumi up inside the container.

I found that the Operator cloned my repository inside /tmp/pulumi-working/<stack-name> I jumped into it and simply ran pupumi up.

Problem...

getcwd() failed: No such file or directory

This error usually appears when the folder you are in does not exist anymore, so I decided to duplicate it and re-run pulumi up.

And voilà, error replicated!

Docker is simply not there..

If I try to build the Docker image by myself directly from the container I receive the following error:

Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

It makes completely sense! If you do not have a Docker Daemon running, you cannot build a Docker image.

The solution

The idea is to add a docker:dind container inside the Pulumi Operator pod to be used as a Docker Daemon.

Starting in 18.09+ the Docker Dind automatically generate TLS certificates. We have to mount a directory accross the containers to share thoses.

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: pulumi-operator-fc266171
  namespace: pulumi
  spec:
    replicas: 1
    selector:
      matchLabels:
        name: pulumi-kubernetes-operator
    template:
      spec:
        volumes:
          - name: dind-storage
            emptyDir: {}
          - name: docker-certs-client
            emptyDir: {}
        containers:
          - name: dind
            image: docker:24-dind
            resources: {}
            volumeMounts:
              - name: dind-storage
                mountPath: /var/lib/docker
              - name: docker-certs-client
                mountPath: /certs/client
              # ^^^^^^
              # Mount volume to share certificates to the operator
            securityContext:
              privileged: true
            # ^^^^
            # Must be defined to run Docker
          - name: pulumi-kubernetes-operator
            image: pulumi/pulumi-kubernetes-operator:v1.12.0
            args:
              - "--zap-level=error"
              - "--zap-time-encoding=iso8601"
            env:
              - name: DOCKER_HOST
                value: 'tcp://localhost:2376'
                     # ^^^^
                     # Define address for remote Docker Daemon 
              - name: DOCKER_CERT_PATH
                value: '/certs'
                     # ^^^^
                     # Define custom certificate directory (mounted volume) 
              - name: DOCKER_TLS_VERIFY
                value: 1
                     # ^^^^
                     # Force Docker client to use TLS 
              - name: WATCH_NAMESPACE
                valueFrom:
                  fieldRef:
                    apiVersion: v1
                    fieldPath: metadata.namespace
              - name: POD_NAME
                valueFrom:
                  fieldRef:
                    apiVersion: v1
                    fieldPath: metadata.name
              - name: OPERATOR_NAME
                value: pulumi-kubernetes-operator
              - name: GRACEFUL_SHUTDOWN_TIMEOUT_DURATION
                value: 5m
              - name: MAX_CONCURRENT_RECONCILES
                value: "10"
            resources: {}
            volumeMounts:
              - name: docker-certs-client
                mountPath: /certs
              # ^^^^^^
              # Mount volume to get Docker certificates

            terminationMessagePath: /dev/termination-log
            terminationMessagePolicy: File
            imagePullPolicy: Always

You can actually check that it works by running docker info inside the Operator Pod.

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.4
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.17.3
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 24.0.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 1677a17964311325ed1c31e2c0a3589ce6d5c30d
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
 Kernel Version: 5.15.90.1-microsoft-standard-WSL2
 Operating System: Alpine Linux v3.18 (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 31.32GiB
 Name: pulumi-operator-anaxago-fc266171-6f896556dc-lxkvp
 ID: 32511430-499e-4fa7-bbce-9b801ffb77ec
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

Tada! 🎉 You can now build images using Pulumi and the Pulumi Operator!

I hope that this Blog Post helped you! If you have any questions, feel free to use the comment section! 💬

Oh and if you want more content like this, follow me:

Build a Blog using Next.JS and DEV.to

Martin PAUCOT — Wed, 17 May 2023 18:04:10 +0000

In this article you will learn how to build a Next.JS blog by fetching your Posts directly from DEV.to.

I received an incredible feedback from my Post Use Notion as a database for your Next.JS Blog thanks from all of you 🙌

I even saw the Post on the front page of daily.dev 😶

Today I wanted to share with you how I built my Personal Blog under an hour by using the DEV.to API.

Let's get started 🔥

1. Create a new Next.JS App

Start by using the create-next-app utility using your favorite package manager.

$ npx create-next-app@latest
$ yarn create next-app
$ pnpm create next-app

Check ✅ for everything! We want linting, typings and obviously the App Router! I also highly recommend to use the src folder.

2. Install dependencies

You will need 9 dependencies:

remark: We will use it to parse our Posts Markdown
remark-html: A Remark plugin to transform our Markdown into HTML
rehype: A library to process and extend HTML
rehype-highlight: A Rehype plugin to plug highlight.js to highlight our code
rehype-slug: A Rehype plugin that adds ids to our Post titles (anchors)
@jsdevtools/rehype-toc: A Rehype plugin that generates a table of content based on our Post titles
rehype-stringify: A Rehype plugin that transform our Rehype output into a String
remark-rehype: A Remark plugin to use Remark and Rehype in symbiose
unified: The library to make easy to use all thoses plugins together

$ npm install remark remark-html rehype rehype-highlight rehype-slug @jsdevtools/rehype-toc rehype-stringify remark-rehype unified

$ yarn add remark remark-html rehype rehype-highlight rehype-slug @jsdevtools/rehype-toc rehype-stringify remark-rehype unified

3. Fetch from DEV.to

DEV.to provide a wonderful Public API that does not require any authentication you can find the official documentation here.

In our case we will need two things:

Fetching our Posts /api/articles?username=<username>
Fetching a specific Post /api/articles/<username>/<post-slug>

Add environment variables

It is a good practice to avoid hardcoding values, in case you want to change your username or open-source your blog.

Add your DEV.to username in .env.local to avoid hardcoding it:

DEVTO_USERNAME="martinp"

Add typings

Let's add some typings to type the response of the DEV.to API in src/lib/devto/types.ts:

// src/lib/devto/types.ts

export type User = {
  user_id: number;
  name: string;
  username: string;
  twitter_username: string | null;
  github_username: string | null;
  website_url: string | null;
  profile_image: string;
  profile_image_90: string;
};

export type Post = {
  type_of: string;
  id: number;
  title: string;
  description: string;
  readable_publish_date: string;
  slug: string;
  path: string;
  url: string;
  comments_count: number;
  collection_id: number | null;
  published_timestamp: string;
  positive_reactions_count: number;
  cover_image: string | null;
  social_image: string;
  canonical_url: string;
  created_at: string;
  edited_at: string;
  crossposted_at: string | null;
  published_at: string;
  last_comment_at: string;
  reading_time_minutes: number;
  tag_list: string[];
  tags: string;
  user: User;
};

export type PostDetails = Post & {
  body_html: string;
  body_markdown: string;
  tags: string[];
};

I manually made thoses types and they maybe does not exactly match the actual API, feel free to update them.

Create the fetching functions

Next create a new file src/lib/devto/fetch.ts, it will contains the functions that will fetch the API. It is a good practice to separate them from your App to make them easily reusable.

// src/lib/devto/fetch.ts

import { notFound } from "next/navigation";
import { Post, PostDetails } from "./types";

export async function fetchPosts(): Promise<Post[]> {
  const res = await fetch(
    `https://dev.to/api/articles?username=${process.env.DEVTO_USERNAME}`,
    {
      next: { revalidate: 3 * 60 * 60 },
    }
  );

  if (!res.ok) notFound();
  return res.json();
}

export async function fetchPost(slug: string): Promise<PostDetails> {
  const res = await fetch(
    `https://dev.to/api/articles/${process.env.DEVTO_USERNAME}/${slug}`,
    {
      next: { revalidate: 3 * 60 * 60 },
    }
  );

  if (!res.ok) notFound();
  return res.json();
}

Notice that we add the parameter revalidate: 3 * 60 * 60. By default the fetch function extended by Next.JS will cache everything. It will make your Blog blazingly fast but we also want to keep our Blog up to date. With this parameter we tell Next.JS to revalidate the cache every 3 hours.

notFound() acts like a return and will show the not-found.tsx page. More information here.

4. Create the render function

Now let's create a function to render the content of your Posts by using Remark, Rehype and all the plugins:

// src/lib/markdown.ts

import toc from "@jsdevtools/rehype-toc";
import rehypeHighlight from "rehype-highlight";
import rehypeSlug from "rehype-slug";
import rehypeStringify from "rehype-stringify";
import remarkRehype from "remark-rehype";
import remarkParse from "remark-parse";
import { unified } from "unified";

export function renderMarkdown(markdown: string): Promise<string> {
  return unified()
    .use(remarkParse)
    .use(remarkRehype)
    .use(rehypeHighlight, { ignoreMissing: true })
    .use(rehypeSlug)
    .use(rehypeStringify)
    .use(toc, {
      headings: ["h1", "h2", "h3"],
    })
    .process(markdown)
    .then((res) => res.toString());
}

5. Create the pages

Now that you have everything in place to fetch your posts, it is time to create the pages!

The Posts page

It can't be simpler, simply use your fetchPosts function and show them:

// src/app/blog/page.tsx

import { fetchPosts } from "@/lib/devto/fetch";

export default async function Page() {
  const posts = await fetchPosts();

  return (
    <div className="grid grid-cols-1 md:grid-cols-2 gap-4 py-8">
      {posts.map((post) => (
        <Link href={`/blog/${post.slug}`}>{post.title}</Link>
      ))}
    </div>
  );
}

The Post page

Create a new page with a dynamic segment for our slug src/app/blog/[slug]/page.tsx.

Use the parameters to fetch the post and use the renderMarkdown function to transform your Markdown into HTML.

You can also add generateMetadata to set the title and the description using the data of your Post.

// src/app/blog/[slug]/page.tsx

import 'highlight.js/styles/github-dark.css'; // Import your favorite highlight.js theme

import { fetchPost, fetchPosts } from "@/lib/devto/fetch";
import { renderMarkdown } from "@/lib/markdown";

export async function generateMetadata({
  params,
}: {
  params: { slug: string };
}) {
  const { title, description } = await fetchPost(params.slug);

  return {
    title,
    description,
  };
}

export default async function Page({ params }: { params: { slug: string } }) {
  const { body_markdown } = await fetchPost(params.slug);

  const content = await renderMarkdown(body_markdown);

  return (
    <>
      <article>
        <div dangerouslySetInnerHTML={{ __html: content }} />
      </article>
    </>
  );
}

Notice that you are calling twice the fetchPost method, so are you fetching twice? No! It uses the cache, you can verify it when running the dev server, you should see cache: HIT.

And you know what is reaaaally cool?
Navigate to the list of your posts and hover the links, you should see in your console your /blog/[slug] pages pre-rendering to predict the user navigation 🔥

6. Going further

Use next-sitemap to generate your Sitemap
Add @vercel/analytics to gather analytics from your blog
Use @tailwindcss/typography to easily style the content of your posts

I hope that this post motivated you to build an incredible blog! Share your work in the comment section! 💬

Oh and if you want more content like this, follow me:

Use Notion as a database for your Next.JS Blog

Martin PAUCOT — Sat, 13 May 2023 16:31:41 +0000

Notion is an extremely powerful tool to manage your content by creating a database you can even add properties to pages: publication date, tags, etc.

In this Post you will learn how to fetch pages from the Notion API and render their content to create a wonderful Next.JS Blog entirely managed with Notion.

1. Create a Notion Database

A Notion Database is a list of pages with defined properties, it provides features to easily manage your content with different type of views (table, calendar, etc).

For the purpose of this guide we will add the following properties:

Title: The title of the post
Date: The date of the post
Status: The status of the post (Not started, Draft, Published)
Created time: The creation date of the post

Do not forget to create your posts and write some content in them!

Feel free to add your own properties and tweak them to your needs. You could for example add a publication date to automaticaly publish at a certain date.

Find the database ID

Later in this guide you will need the ID of your database. You will find it in the URL: https://www.notion.so/myworkspace/50b6156388e445eaaca3a3599d6f7ade

2. Get a Notion Token

Create an Integration

In order to interact with the Notion API you will need an Internal Integration Token aka. Notion Token.

Head over the following link to create a new Notion Integration. In our case we will only read data, you should only add the Read capacity.

When your integration is created you will have an Internal Integration Token. Save it and keep it safe, it will be the "Notion token" that you will use to authenticate to the API.

Authorize the integration to your databases

You must explicitly give the permission to your integration to query your databases.

Click on the ••• in the top right corner of your database, then on Add connection and select your Integration.

To avoid giving access to each of your databases, you can add the integration to a parent page.

3. Setup the project

Let's install the required dependencies. We are going to use four libraries:

@notionhq/client The official Notion Javascript SDK
@notion-render/client A library to transform Notion Blocks (page content) into HTML
@notion-render/hljs-plugin A plugin to highlight your code blocks
@notion-render/bookmark-plugin A plugin to fetch website metadata to render bookmarks



$ yarn add @notionhq/client @notion-render/client @notion-render/hljs-plugin @notion-render/bookmark-plugin
# Or
$ npm install @notionhq/client @notion-render/client @notion-render/hljs-plugin @notion-render/bookmark-plugin

Then store your Internal Integration Token and the database Id into your .env.local file so you can access it later.



NOTION_TOKEN="secret_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
NOTION_DATABASE_ID="xxxxxxxxxxxxxxxxxxxxxxx"

4. Create the Post Page

Create the Notion Client

Create a new file lib/notion.ts we will add inside the functions we need to fetch our posts.



import "server-only";

import { Client } from "@notionhq/client";
import React from "react";
import {
  BlockObjectResponse,
  PageObjectResponse,
} from "@notionhq/client/build/src/api-endpoints";

export const notion = new Client({
  auth: process.env.NOTION_TOKEN,
});

export const fetchPages = React.cache(() => {
  return notion.databases.query({
    database_id: process.env.NOTION_DATABASE_ID!,
    filter: {
      property: "Status",
      select: {
        equals: "Published",
      },
    },
  });
});

export const fetchPageBySlug = React.cache((slug: string) => {
  return notion.databases
    .query({
      database_id: process.env.NOTION_DATABASE_ID!,
      filter: {
        property: "Slug",
        rich_text: {
          equals: slug,
        },
      },
    })
    .then((res) => res.results[0] as PageObjectResponse | undefined);
});

export const fetchPageBlocks = React.cache((pageId: string) => {
  return notion.blocks.children
    .list({ block_id: pageId })
    .then((res) => res.results as BlockObjectResponse[]);
});

You can notice two things:

- `import 'server-only';`

This line make sure that the file never get imported by the client to avoid leaking your Notion Token.

- `React.cache`

Next.JS provide an extremely good caching system with the fetch() function but we can not benefit from it as we are using the Notion JS SDK.

Instead we can use React.cache, a powerful method that will returns the same result if we call our function with the same parameters.

Create the page

Create a page with a dynamic segment [slug]. Inside we will fetch our pages so it must be a Server Component:



// app/blog/[slug]/page.tsx

import { fetchPageBlocks, fetchPageBySlug } from "@/lib/notion";

import { notFound } from "next/navigation";

export default async function Page({ params }: { params: { slug: string } }) {

  const post = await fetchPageBySlug(params.slug);

  if (!post) notFound();

const content = await fetchPageBlocks(post.id);

return <></>;

}

Render the page content



import { fetchPageBlocks, fetchPageBySlug, notion } from "@/lib/notion";

import bookmarkPlugin from "@notion-render/bookmark-plugin";

import { NotionRenderer } from "@notion-render/client";

import hljsPlugin from "@notion-render/hljs-plugin";

import { notFound } from "next/navigation";

export default async function Page({ params }: { params: { slug: string } }) {

  const post = await fetchPageBySlug(params.slug);

  if (!post) notFound();

const blocks = await fetchPageBlocks(post.id);

const renderer = new NotionRenderer({

    client: notion,

  });

renderer.use(hljsPlugin());

  renderer.use(bookmarkPlugin());

const html = await renderer.render(...blocks);

return <div dangerouslySetInnerHTML={{ __html: html }}></div>;

}

Next steps

Import your favorite highlight.js theme
Import the Notion theme from @notion-render/client/sass/theme.scss
Create a theme with your own branding
Use generateStaticParams to generate pages at build time
Use draftMode to preview your Post not published yet

How to properly use third-party libraries with React Server Components

Martin PAUCOT — Sat, 13 May 2023 06:19:07 +0000

With the new Next.JS 13 App Router, arrived React Server Components. When following the best practices of this new version, we have to think server first but it can be easy to get tangled in server and client components when working with third-party libraries.

What is a React Server Component

A React Server Component is rendered server-side. The generated HTML is then sent to the browser to hydrate the updated part of your app.

PHP Developers will recognize this pattern!

There are a lot of benefits of using React Server Components:

Drastically decrease the amount of Javascript sent to the browser
React Server Components can be entirely cached
Ability to fetch data directly from your component

The problem with React Server Components

If you try to use React Hooks (useState, useEffects, etc) or simply listen to an event (onClick, onChange). You will receive an error similar to this one:

You're importing a component that needs useState. It only works in a Client Component but none of its parents are marked with "use client", so they're Server Components by default.

This is completely normal as the server is not able to listen to events happening in the browse.

To fix the issue you have to add 'use client'; on top of your file to tell Next.JS that everything inside the file is "client-side".

But watchout! Do not add 'use client'; everywhere as if it was a patch. The idea is to keep most of your components server side.

The problem with UI Libraries

You just installed your favorite UI library, and start using it:

// src/app/page.tsx
import { Button } from "@chakra-ui/react";

export default async function Page() {
  const post = await fetchPost();
  return <Button>{post.title}</Button>;
}

You received the same error You're importing a component that needs useState but you do not want to change your page into a Client Component as you need to fetch data.

So you decide to create a "component-in-the-middle" that will tell Next.JS that the Button imported from @chakra-ui/react is a Client Component:

// src/app/Button.tsx
"use client";
import { Button as ChakraButton } from "@chakra-ui/react";

export default function Button({ children }: { children?: React.ReactNode }) {
  return <ChakraButton>{children}</ChakraButton>;
}

// src/app/page.tsx
import Button from "./Button";

export default async function Page() {
  const post = await fetchPost();
  return <Button>{post.title}</Button>;
}

It works! But for this case we only forwarded the children, we want every props. Let's do it with a much simpler solution:

// src/app/Button.tsx
"use client";
import { Button as ChakraButton } from "@chakra-ui/react";

const Button = ChakraButton;

export default Button;

But it means that we have to do the same thing for every single component that we use, is there any better solution?

"use client";
export * from "@chakra-ui/react";

import { Button } from "./ChakraUI";

export default function Page() {
  const post = await fetchPost();
  return <Button>{post.title}</Button>;
}

What about "dot notation" like `framer-motion`

Unfortunately there is no magic solution currently.
You have to export each component:

"use client";

import { motion } from "framer-motion";

export const MotionDiv = motion.div;
export const MotionSpan = motion.span;

Conclusion

Properly using React Server Component can drastically increase your app performances but it is tricky to keep a good component tree without getting tangled between server and client.

Currently most of the libraries does not yet export using 'use client'; directive so you still have to add it ourself.

Stop Nesting your code

Martin PAUCOT — Fri, 12 May 2023 09:39:22 +0000

I am what we call a Never Nester and in this Post I will show you why and how to become one of us.

What is Nesting?

When using conditions, for-loops or while-loops we add one layer of indentation of the wrapped code. Some languages requires it and some no. But at the end it is essential for readability.

In the following code that I purposely made unappealing for this Post, we can see that we have up to three layers:

The condition matching the email
The condition checking for an existing user
The try catch for the creation of our user

async function registerUser(email: string, plainPassword: string) {
  if (email.match(/^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$/)) {
    let user = await findUserByEmail(email);
    if (!user) {
      const rounds = process.env.BCRYPT_ROUNDS;
      if (rounds) {
        try {
          const password = await bcrypt.hash(plainPassword, 10);
          user = await createUser(email, password);
          return user;
        } catch (e) {
          throw new Error("Error while creating user");
        }
      } else {
        throw new Error("Environment variable BCRYPT_ROUNDS not found");
      }
    } else {
      throw new Error("User already exists");
    }
  } else {
    throw new Error("Not a valid email");
  }
}

The problem of Nesting

The more layers you have, the harder it is to read.

When reading nested code we have to keep in mind in what context we are, in what conditions or loop we are.

And when using if {} else {} we have to go back and forth to understand what happens if the condition does not match while keeping the context when reading.

How to Denest

Invert your code

We usually write our code the way we think it:

"I want to be sure that my email is exact"
"I want to be sure that I do not have a user with the same email"

But with this process we handle the negative case at the end of our code in the else. Instead we should imagine our code this way:

"I do not want a user with a wrong email"
"I do not want to have a user with the same email"

The goal of this process is to return or throw as early as possible:

async function registerUser(email: string, plainPassword: string) {
  if (!email.match(/^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$/)) {
    throw new Error("Not a valid email");
  }

  let user = await findUserByEmail(email);
  if (user) {
    throw new Error("User already exists");
  }

  const rounds = process.env.BCRYPT_ROUNDS;
  if (!rounds) {
    throw new Error("Environment variable BCRYPT_ROUNDS not found");
  }

  try {
    const password = await bcrypt.hash(plainPassword, 10);
    user = await createUser(email, password);
    return user;
  } catch (e) {
    throw new Error("Error while creating user");
  }
}

We now have only one layer by simply inverting our conditions.

Extract your code

By extracting we do not only improve the readability of our code, but also its reusability and abstraction.

To find what we should extract we have to ask ourself what are the minimum dependencies required by the code I'm writing.

The method is to replace most of the by a and removing most of the subjects we can.

For example:

"I want to hash the password of the user that I am registering using the environment variable BCRYPT_ROUNDS that can be undefined"

Become:

"I want to hash a password using the environment variable BCRYPT_ROUNDS that can be undefined"

We know for sure that we will always hash the password everytime a user is created, we can also move this part in the createUser function to avoid future duplicated code.

async function createUser(email, password): Promise<User> {
  return db.users.insert({
    email,
    password: hashPassword(password),
  });
}

async function hashPassword(password: string) {
  const rounds = process.env.BCRYPT_ROUNDS;
  if (!rounds) throw new Error("Environment variable BCRYPT_ROUNDS not found");
  return bcrypt.hash(password, 10);
}

async function registerUser(email: string, plainPassword: string) {
  if (!email.match(/^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$/)) {
    throw new Error("Not a valid email");
  }

  let user = await findUserByEmail(email);
  if (user) {
    throw new Error("User already exists");
  }

  return createUser(email, plainPassword);
}