Víctor Adrián

Posted on Apr 7, 2018 • Originally published at lobotuerto.com on Apr 6, 2018

Building a JSON API with Phoenix 1.3 and Elixir

#tutorial #elixir #phoenix #api

You can check the latest updated version of this article at lobotuerto's notes - Building a JSON API with Phoenix 1.3 and Elixir.

It also includes a content table for easy navigation over there. ;)

Do you have a Rails background?

Are you tired of looking at outdated or incomplete tutorials on how to build a JSON API using Elixir and Phoenix?

Then, read on my friend!

Introduction

I have found there are mainly two types of tutorials one should write:

Focused, scoped tutorials.
Full step-by-step tutorials.

Scoped, focused tutorials, should be used to explain techniques, like this one: Fluid SVGs with Vue.js.

But, fully-integrated tutorials should be used to teach about new tech stacks.

Going from zero to fully working prototype without skipping steps.

With best practices baked-in, presenting the best libreries available for a given task.

I really like tutorials that take this holistic approach.

So, this won’t just be about how to generate a new Phoenix API only app. That’s easy enough, you just need to pass the --no-brunch --no-htmlto mix phx.new.

This tutorial is about creating a small, but fully operational JSON API for web applications.

To complement your API, I recommend Vue.js on the frontend:
Quickstart guide for a new Vue.js project.

What we’ll do:

Create a new API-only Phoenix application —skip HTML and JS stuff.
Create a User schema module (model) and hash its password —because storing plain text passwords in the database is just wrong.
Create a Users endpoint —so you can get a list of, create or delete users!
CORS configuration —so you can use that frontend of yours that runs on another port / domain.
Create a Sign in endpoint —using session based authentication. I can do a JWT one later if enough people are interested.

Let me be clear about something…

I’m just starting with Elixir / Phoenix, if there are any omissions or bad practices, bear with me, notify me and I’ll fix them ASAP.

This is the tutorial I wish I had available when I was trying to learn about how to implement a JSON API with Elixir / Phoenix, but I digress.

Prerequisites

Install Elixir

We will start by installing Erlang and Elixir using the asdf version manager —using version managers is a best practice in development environments.

How to install Elixir in Manjaro Linux.

Install Hex

Let’s take on Hex the package manager now:

mix local.hex

You can now print some info about your Elixir stack with:

mix hex.info

You’ll see something along the lines of:

Hex: 0.17.3
Elixir: 1.6.4
OTP: 20.3.2

Built with: Elixir 1.5.3 and OTP 18.3

Install Phoenix

Now let’s install the framework:

mix archive.install https://github.com/phoenixframework/archives/raw/master/phx_new.ez

Install PostgreSQL

PostgreSQL is the default database for new Phoenix apps, and with good reason: It’s a solid, realiable, and well engineered relational DB.

How to install PostgreSQL in Manjaro Linux.

About REST clients

You might need to get a REST client so you can try out your API endpoints.

The two most popular ones seem to be Postman and Advanced Rest Client I tested both of them and I can say liked neither —at least on their Chrome app incarnations— as one didn’t display cookie info, and the other didn’t send declared variables on POST requests. ¬¬

In any case if you want to try them out:

You can get Postman here.
You can get ARC here.

If you are using a web frontend library like Axios, then your browser’s developer tools should be enough:

If you go with Axios dont’ forget to pass the configuration option withCredentials: true, this will allow the client to send cookies along when doing CORS requests.

Or you can just use good ol’ curl it works really well!

I’ll show you some examples on how to test out your endpoints from the CLI.

Create a new API-only Phoenix application

Generate the app files

In your terminal:

mix phx.new my-app --app my_app --module MyApp --no-brunch --no-html

From the command above:

You’ll see my-app as the name for the directory created for this application.
You’ll see my_app used in files and directories inside my-app/lib e.g. my_app.ex.
You’ll find MyApp used everywhere since it’s the main module for your app.

For example in my-app/lib/my_app.ex:

defmodule MyApp do
  @moduledoc """
  MyApp keeps the contexts that define your domain
  and business logic.

  Contexts are also responsible for managing your data, regardless
  if it comes from the database, an external API or others.
  """
end

Create the development database

If you created a new DB user when installing PostgreSQL , add its credentials toconfig/dev.exs and config/test.exs. Then execute:

cd my-app
mix ecto.create

NOTE:

You can drop the database for the dev environment with:

mix ecto.drop

If you’d like to drop the database for the test environment, you’d need to:

MIX_ENV=test mix ecto.drop

Start the development server

From your terminal:

mix phx.server

Visit http://localhost:4000 and bask in the glory of a beautifully formatted error page. :)

Don’t worry though, we’ll be adding a JSON endpoint soon enough.

Errors in JSON for 404s and 500s

If you don’t like to see HTML pages when there is an error and instead want to receive JSONs, set debug_errors to false in your config/dev.ex, and restart your server:

config :my_app, MyAppWeb.Endpoint,
  # ...
  debug_errors: false,
  # ...

Now, visiting http://localhost:4000 yields:

{ "errors": { "detail": "Not Found" } }

User schema (model)

We'll be generating a new User schema (model) inside an Auth context.

Contexts in Phoenix are cool, they serve as API boundaries that let you
organize your application code in a better way.

Generate the Auth context and User schema module

mix phx.gen.context Auth User users email:string:unique password:string is_active:boolean

From above:

Auth is the context’s module name.
User is the schema’s module name.
users is the DB table’s name.
After that comes some field definitions.

The migration generated from the command above looks like this:

defmodule MyApp.Repo.Migrations.CreateUsers do
  use Ecto.Migration

  def change do
    create table(:users) do
      add :email, :string#, null: false
      add :password, :string
      add :is_active, :boolean, default: false, null: false

      timestamps()
    end

    create unique_index(:users, [:email])
  end
end

Adjust it to your liking —i.e. not allowing null for emails— then run the new migration:

mix ecto.migrate

If you want to read some info about this generator, execute:

mix help phx.gen.context

Hash a user’s password on saving

Add a new dependency to mix.exs:

defp deps do
    [
      # ...
      {:bcrypt_elixir, "~> 1.0"}
    ]
  end

This is Bcrypt, we will use it to hash the user’s password before saving it; so we don’t store it as plain text inside the database.

Fetch the new app dependencies with:

mix deps.get

Change lib/my_app/auth/user.ex to look like this:

defmodule MyApp.Auth.User do
  # ...
  def changeset(user, attrs) do
    user
    |> cast(attrs, [:email, :password, :is_active])
    |> validate_required([:email, :password, :is_active])
    |> unique_constraint(:email)
    |> hash_user_password()
  end

  defp hash_user_password(%Ecto.Changeset{valid?: true, changes: %{password: password}} = changeset) do
    change(changeset, password: Bcrypt.hash_pwd_salt(password))
  end

  defp hash_user_password(changeset) do
    changeset
  end
end

Notice the call and definitions of hash_user_password/1.

What this does is run the changeset through that function, and if the changeset happens to have a password key, it’ll use Bcrypt to hash it.

Running Bcrypt.hash_pwd_salt("hola") would result in something like"$2b$12$sI3PE3UsOE0BPrUv7TwUt.i4BQ32kxgK.REDv.IHC8HlEVAkqmHky".

That strange looking string is what ends up being saved in the database instead of the plain text version.

Fix the tests

Run the tests for your project with:

mix test

Right now they will fail with:

1) test users create_user/1 with valid data creates a user (MyApp.AuthTest)
     test/my_app/auth/auth_test.exs:32
     Assertion with == failed
     code: assert user.password() == "some password"
     left: "$2b$12$PUK73EqrvBTuOi2RiVrkOexqiVS.wIwbOtyR0EtzQLpbX6gaka8T2"
     right: "some password"
     stacktrace:
       test/my_app/auth/auth_test.exs:36: (test)

  2) test users update_user/2 with valid data updates the user (MyApp.AuthTest)
     test/my_app/auth/auth_test.exs:43
     Assertion with == failed
     code: assert user.password() == "some updated password"
     left: "$2b$12$cccPJfQD3seaBc8pHX8cJO/549lojlAjNNi/qo9QY0K7a7Zm5CaNG"
     right: "some updated password"
     stacktrace:
       test/my_app/auth/auth_test.exs:49: (test)

That’s because of the change we just made, the one that hashes the password.

But this is easily fixed by changing those assertions to use Bcrypt.verify_pass/2.

Open the file test/my_app/auth/auth_test.exs and change these lines:

# ...
test "create_user/1 with valid data creates a user" do
  # ...
  assert user.password == "some password"
end
# ...
test "update_user/2 with valid data updates the user" do
  #...
  assert user.password == "some updated password"
end
# ...

To:

# ...
test "create_user/1 with valid data creates a user" do
  # ...
  assert Bcrypt.verify_pass("some password", user.password)
end
# ...
test "update_user/2 with valid data updates the user" do
  # ...
  assert Bcrypt.verify_pass("some updated password", user.password)
end
# ...

Now mix test should yield no errors.

Users endpoint

Generate a new JSON endpoint

Let’s generate the users JSON endpoint, since we already have the Auth context and User schema available, we will pass the --no-schema and --no-context options.

mix phx.gen.json Auth User users email:string password:string is_active:boolean --no-schema --no-context

Fix the tests

Now, if you try and run your tests you’ll see this error:

== Compilation error in file lib/my_app_web/controllers/user_controller.ex ==
** (CompileError) lib/my_app_web/controllers/user_controller.ex:18: undefined function user_path/3

It’s complaining about a missing user_path/3 function.

You need to add a resources line to lib/my_app_web/router.ex.

Declaring a resource in the router will make some helpers available for its controller —i.e. user_path/3.

defmodule MyAppWeb.Router do
  # ...
  scope "/api", MyAppWeb do
    pipe_through :api

    resources "/users", UserController, except: [:new, :edit]
  end
end

Nevertheless, tests will still complain.

To fix them we need to make two changes:

Comment out the password line in lib/my_app_web/views/user_view.ex:

defmodule MyAppWeb.UserView do
  # ...
  def render("user.json", %{user: user}) do
    %{id: user.id,
      email: user.email,
      # password: user.password,
      is_active: user.is_active}
  end
end

We don’t need —nor should we— be sending the hashed password in our responses.

Since we won’t be receiving the hashed password in the response, comment out these password lines in test/my_app_web/controllers/user_controller_test.exs:

# ...
assert json_response(conn, 200)["data"] == %{
  "id" => id,
  "email" => "some email",
  "is_active" => true
  # "password" => "some password"
}
# ...
assert json_response(conn, 200)["data"] == %{
  "id" => id,
  "email" => "some updated email",
  "is_active" => false
  # "password" => "some updated password"
}
# ...

Tests should be fine now.

Create a couple of users

Using IEx

You can run your app inside IEx (Interactive Elixir) —this is akin to rails console— with:

iex -S mix phx.server

Then create a new user with:

MyApp.Auth.create_user(%{email: "asd@asd.com", password: "qwerty"})

Using curl

If you have curl available in your terminal, you can create a new user through your endpoint using something like:

curl -H "Content-Type: application/json" -X POST \
-d '{"user":{"email":"some@email.com","password":"some password"}}' \
http://localhost:4000/api/users

CORS configuration

You’ll need to configure this if you plan on having your API and frontend on different domains.

If you don’t know what CORS is, have a look at this: Cross-Origin Resource Sharing (CORS).

That said, here we have two options:

CorsPlug.
Corsica.

I’ll be using CorsPlug in this tutorial, but if you need more features for configuring your CORS requests —or want a more strict libray, try Corsica out.

Add this dependency to mix.exs:

defp deps do
    [
      # ...
      {:cors_plug, "~> 1.5"}
    ]
  end

Fetch new dependencies with:

mix deps.get

Add plug CORSPlug to lib/my_app_web/endpoint.ex:

defmodule MyAppWeb.Endpoint do
  # ...
  plug CORSPlug, origin: "http://localhost:8080"

  plug MyAppWeb.Router
  # ...
end

You can pass a list to origin, as well as a regular expression, or justplug CORSPlug to accept all origins —since origin: "*" is the default.

In my case, the rule above will accept CORS requests from a Vue.js frontend —Vue.js development servers go up on port 8080 by default.

Simple authentication

Verify a user’s password

Let’s add some functions to the lib/my_app/auth/auth.ex file to verify a user’s password:

defmodule MyApp.Auth do
  # ...
  def authenticate_user(email, password) do
    query = from u in User, where: u.email == ^email
    query |> Repo.one() |> verify_password(password)
  end

  defp verify_password(nil, _) do
    {:error, "Wrong email or password"}
  end

  defp verify_password(user, password) do
    case Bcrypt.verify_pass(password, user.password) do
      true -> {:ok, user}
      false -> {:error, "Wrong email or password"}
    end
  end
end

sign_in endpoint

Then add a new sign_in endpoint to lib/my_app_web/router.ex:

defmodule MyAppWeb.Router do
  # ...
  scope "/api", MyAppWeb do
    # ...
    resources "/users", UserController, except: [:new, :edit]
    post "/users/sign_in", UserController, :sign_in
  end
end

sign_in controller function

Finally add the sign_in function to lib/my_app_web/controllers/user_controller.ex:

defmodule MyAppWeb.UserController do
  # ...
  def sign_in(conn, %{"email" => email, "password" => password}) do
    case MyApp.Auth.authenticate_user(email, password) do
      {:ok, user} ->
        conn
        |> put_status(:ok)
        |> render(MyAppWeb.UserView, "sign_in.json", user: user)

      {:error, message} ->
        conn
        |> put_status(:unauthorized)
        |> render(MyAppWeb.ErrorView, "401.json", message: message)
    end
  end
end

Define sing_in.json and 401.json views

In lib/my_app_web/user_view.ex add this:

defmodule MyAppWeb.UserView do
  # ...
  def render("sign_in.json", %{user: user}) do
    %{data:
      %{user:
        %{id: user.id, email: user.email}}}
  end
end

In lib/my_app_web/error_view.ex add this:

defmodule MyAppWeb.ErrorView do
  # ...
  def render("401.json", %{message: message}) do
    %{errors: %{detail: message}}
  end
  # ...
end

You can try the sign_in endpoint now.

Try out your new endpoint with curl

Let’s send some POST requests to http://localhost:4000/api/users/sign_in.

Good credentials

curl -H "Content-Type: application/json" -X POST \
-d '{"email":"asd@asd.com","password":"qwerty"}' \
http://localhost:4000/api/users/sign_in -i

You’ll receive a 200 with:

{
  "data": {
    "user": { "id": 1, "email": "asd@asd.com" }
  }
}

Bad credentials

curl -H "Content-Type: application/json" -X POST \
-d '{"email":"asd@asd.com","password":"not the right password"}' \
http://localhost:4000/api/users/sign_in -i

You’ll get a 401 and:

{ "errors": { "detail": "Wrong email or password" } }

Sessions

Add plug :fetch_session to your :api pipeline in lib/my_app_web/router.ex:

defmodule MyAppWeb.Router do
  # ...
  pipeline :api do
    plug :accepts, ["json"]
    plug :fetch_session
  end
  # ...
end

Save authentication status

Now let’s modify out sign_in function in lib/my_app_web/controllers/user_controller.ex:

defmodule MyAppWeb.UserController do
  # ...
  def sign_in(conn, %{"email" => email, "password" => password}) do
    case MyApp.Auth.authenticate_user(email, password) do
      {:ok, user} ->
        conn
        |> put_session(:current_user_id, user.id)
        |> put_status(:ok)
        |> render(MyAppWeb.UserView, "sign_in.json", user: user)

      {:error, message} ->
        conn
        |> delete_session(:current_user_id)
        |> put_status(:unauthorized)
        |> render(MyAppWeb.ErrorView, "401.json", message: message)
    end
  end
end

Protect a resource with authentication

Modify your lib/my_app_web/router.ex to look like this:

defmodule MyAppWeb.Router do
  use MyAppWeb, :router

  pipeline :api do
    plug :accepts, ["json"]
    plug :fetch_session
  end

  pipeline :api_auth do
    plug :ensure_authenticated
  end

  scope "/api", MyAppWeb do
    pipe_through :api
    post "/users/sign_in", UserController, :sign_in
  end

  scope "/api", MyAppWeb do
    pipe_through [:api, :api_auth]
    resources "/users", UserController, except: [:new, :edit]
  end

  # Plug function
  defp ensure_authenticated(conn, _opts) do
    current_user_id = get_session(conn, :current_user_id)

    if current_user_id do
      conn
    else
      conn
      |> put_status(:unauthorized)
      |> render(MyAppWeb.ErrorView, "401.json", message: "Unauthenticated user")
      |> halt()
    end
  end
end

As you can see we added a new pipeline called :api_auth that’ll run requests through a new :ensure_authenticated plug function.

We also created a new scope "/api" block that pipes its requests through :apithen through :api_auth and moved resources "/users" inside.

Isn’t it amazing the way you can define this stuff in Phoenix?!

Composability FTW!

Endpoint testing with curl

Try to request a protected resource, like /api/users with:

curl -H "Content-Type: application/json" -X GET \
http://localhost:4000/api/users \
-c cookies.txt -b cookies.txt -i

You’ll get:

{ "errors": { "detail": "Unauthenticated user" } }

Let’s login with:

curl -H "Content-Type: application/json" -X POST \
-d '{"email":"asd@asd.com","password":"qwerty"}' \
http://localhost:4000/api/users/sign_in \
-c cookies.txt -b cookies.txt -i

You’ll get:

{
  "data": {
    "user": { "id": 1, "email": "asd@asd.com" }
  }
}

Now, try requesting that protected resource again:

curl -H "Content-Type: application/json" -X GET \
http://localhost:4000/api/users \
-c cookies.txt -b cookies.txt -i

You’ll see:

{
  "data": [
    { "is_active": false, "id": 1, "email": "asd@asd.com" },
    { "is_active": false, "id": 2, "email": "some@email.com" }
  ]
}

Success!

Bonus section

Customize your 404s and 500s JSON responses

In lib/my_app_web/views/error_view.ex:

defmodule MyAppWeb.ErrorView do
  # ...
  def render("404.json", _assigns) do
    %{errors: %{detail: "Endpoint not found!"}}
  end

  def render("500.json", _assigns) do
    %{errors: %{detail: "Internal server error :("}}
  end
  # ...
end

Introduction

Prerequisites

Install Elixir

Install Hex

Install Phoenix

Install PostgreSQL

About REST clients

Create a new API-only Phoenix application

Generate the app files

Create the development database

Start the development server

Errors in JSON for 404s and 500s

User schema (model)

Generate the Auth context and User schema module

Hash a user’s password on saving

Fix the tests

Users endpoint

Generate a new JSON endpoint

Fix the tests

Create a couple of users

Using IEx

Using curl

CORS configuration

Simple authentication

Verify a user’s password

sign_in endpoint

sign_in controller function

Define sing_in.json and 401.json views

Try out your new endpoint with curl

Good credentials

Bad credentials

Sessions

Save authentication status

Protect a resource with authentication

Endpoint testing with curl

Bonus section

Customize your 404s and 500s JSON responses

Links

Read next

Cohesion and Coupling in Javascript

Add Step-up Authentication Using Angular and NestJS

Top 5 APIs that can improve your SEO game

Console.WriteLine in C#: How To Use It