DEV Community: Josh

Taming Github OAuth integrations in Phoenix with the help of Ueberauth and Guardian (but then actually no just Ueberauth)

Josh — Wed, 15 Sep 2021 19:54:43 +0000

The problem comes into focus
Caturday afternoon accomplishments
Creating your app in Github: an intermission
This Tutorial: The Code Parts
1. Package dependencies
2. YourAppWeb.Router
3. Controllers, Views, and Templates
  1. In which Ueberauth plays gatekeeper in more ways than one
  2. Not done quite yet…
🎵 Going auth the rails on a crazy train 🎵
1. UserAuth
  1. .current_user/1
  2. .log_in/2
  3. .log_out/2
  4. .require_signin/2
  5. .fetch_current_user/2
2. YourAppWeb.Endpoint
3. config/*.exs
You made it!

You're working on your super-slick Phoenix app, and early on, determine that it wouldn't make any sense for you to build the thing without integrating with GitHub. After all, your app is for developers, and developers do use GitHub, unless they're using GitLab or BitBucket, in which case they still use something, and you're gonna get around to building out those OAuth use cases too, it's just first let's focus on the biggest piece of the pie, because you have no money, and pie sounds way more appetizing than malnourishment right now.

So you go online and check around for phoenix github oauth integration and oh, look, it's a blog post from thoughtbot on that very subject. You remember your interview with thoughtbot in late 2017 fondly, and though your nagging worry that candidly sharing your plans for that weekend when they asked was possibly the reason they passed on making you an offer,¹ you hold yourself above grudges, and definitely harbor nothing of the sort against their dumb stupid faces.²

As you read through the post you discover, to your annoyance, that not all of the information you'll need in order to build what you're trying to build is given to you in one solid block of practical information and code examples. You also glean from the words "Part 4" in the title that the practical information you crave might not even be contained in one solid blog post and that you are about to scavenge, ratlike, through no fewer than three (3) other posts in order to have the full working mechanism that is user authentication in Phoenix using Ueberauth + Guardian.

You acquiesce, and trudge through the four-post walkthrough, acquainting yourself with Ueberauth and Guardian along the way. It bugs you that a number of the settings you end up configuring for Guardian seem both esoteric ("I need to implement resource_for_token, which takes a resource and throws away _claims, but... then I retrieve the resource from claims later on? In resource_from_claims? Why is it in the map under the key "sub"?")³, yet somehow redundant ("I have to create YourApp.Authentication.Pipeline module that uses Guardian.Plug.Pipeline, then give it references to the otp_app it should already be part of, a custom ErrorHandler, and module it's already namespaced in?"), but it's the most popular Hex package that promises OAuth 2.0 support, and it even uses those hot new JWTs that're all the rage these days. Once everything's in place, you feel more or less like you've successfully implemented an OAuth registration workflow, with all of the "bells and whistles" of session persistence, logging out, and even logging back in again, all painlessly slotted into your middleware workflow through the magic of Plug's pipelines and plugs.

The problem comes into focus

Until, of course, you try to configure the session's expiration time – which received no (0) attention in any of the four (4) posts you were following along with – and you discover that not only do Guardian's configuration settings resemble the esoteric scratchings of druid runes, Guardian's documentation gives you about as much insight into their proper usage, as you personally have into the proper usage of esoteric druid runes, and the task of implementing the session expiration gets derailed almost as soon as you begin flipping through Guardian's hexdocs pages.

During a brief out-of-body experience, you chuckle about the schadenfreude in which you find yourself currently steeped, and once back in your body you commence spelunking through Guardian's code, hoping in vain that it offers some better insights when how it works is laid out in front of you. But you stop when your subconscious rings a small bell in the middle side of your brain, and the sudden joy of epiphany informs you that parts of your /lib folder now bear a striking resemblance to the code blocks you saw reading through the section of Chris McCord, José Valim, and Bruce Tate's book, Programming Phoenix ≥ 1.4 about building an authentication and authorization pipeline for your app. In fact, you're pretty sure that just a few small adjustments would be all you'd need before you could mercilessly tear Guardian out of your package dependencies completely and still see your code run perfectly fine.⁴

You get the coffee brewing, put on your Getting Stuff Done music, and hammer out code like a cat in a GIF.

Literally like any/all of these fine feline folx

Caturday afternoon accomplishments

After an hour or two⁵ ⁶, you lean back at your desk, sip your last bit of coffee, and admire the mature tone of your expertly-crafted commit message, signaling your triumph over user session expiration:

commit 144a2a5ca5104b914c8ffc264b2cef79bd38e781 (HEAD -> github_oauth_workflow)
Author: You <you@you.you>
Date:   Sat Apr 17 15:03:04 2021 +0000

    Suck it, backstabbing Guardian jerk. DELETED!!

    Also session expirationisimplemented or something

…but especially signaling your triumph over Guardian.

One git push, pull request, and rebase-merge later, and your code is there on your repo's main branch, proud and ready to guide users through an absurdly streamlined authorization workflow from start to finish. And all in all, it's not really as complex as that thoughtbot blog had you worried it'd be.

Of course, the prelude to all of this was setting up your app in Github, which you did once for the dev environment, and once again for staging. You'll do it one more time once you've made your first push to prod, but as to when that may come to fruition, qui peut dire? ⁷ ¯\_(ツ)_/¯

Creating your app listing in Github (An Intermission)

On github.com, go to your "Settings" page
Find the "Developer Settings" link in the sidebar
Click "Github Apps", then "New Github App"
Fill out a good name for your app. Something descriptive, but also very opaque or even random, if you're still operating in stealth mode, and then also denoting (to yourself, mainly) that this is the dev version of the Github listing; something against which you can prototype and test new features with fuzzed data, or data from your own personal projects. Something like… your_app-dev.
Run brew install --cask ngrok (or your preferred local-tunneling daemon)⁸ and pretend you did this step ages ago!
Now run ngrok http --bind-tls true 4000 and copy the ngrok.io URL from the "Forwarding" row of the dashboard now displaying in your Terminal
Paste that URL into the "Homepage URL" field in github's New App form
Paste it into the "Callback URL" field below that, and add /auth/github/callback to the end of it in that field
Check the box for "Request user authorization (OAuth) during installation"
(This doesn't pertain to OAuth, directly, but will come up for you later while you're building something else, so just) paste your ngrok.io URL into the "Webhook URL" field
run mix phx.gen.secret | tee .scratchpaper | pbcopy in your Terminal. This generates a strong, random 32-character long string, saves it to a file at the base of your project called .scratchpaper, and then immediately pushes it to your clipboard (Linux users, you'll want to use xclip -selection clipboard [which you'll want to install { unless you're not running the X window system, then idk what to tell you ¯\_(ツ)_/¯ } from your package manager] in place of pbcopy)
Paste that secret into the "Webhook Secret" field on Github's New App page
Select the permissions you'll need in order to support your app's features beyond authorization
1. Make sure you're subscribed to their webhook events
2. And that you add that webhook_secret to your config/dev.secret.exs or .envrc (If you copied something else to your clipboard before getting to this step, running pbcopy < .scratchpaper will put it back there for you)!

Oof. That was a fun detour. But it was necessary!

This tutorial: The Code Parts

Dependencies

You begin with the dependencies in your mixfile:

# mix.exs

defp deps do
  # ...
  {:ueberauth, "~> 0.6.3"},
  {:ueberauth_github, "~> 0.8.0"}
  # ...
end

Router

Then you define the pipelines and routes you'll use for authentication, and for authorization-restricted access.

# lib/your_app_web/router.ex

import YourAppWeb.UserAuth, only [fetch_current_user: 2, require_signin: 2]

pipeline :browser do
  plug :accepts, ["html"]
  plug :fetch_session
  plug :fetch_live_flash
  plug :protect_from_forgery
  plug :put_secure_browser_headers
  plug :fetch_current_user
end

pipeline :authentication_required do
  plug :require_signin
end

scope "/", YourAppWeb do
  pipe_through :browser

  get "/register", AuthenticationController, :register
  get "/login", AuthenticationController, :login
  get "/logout", AuthenticationController, :logout      # not strictly RESTful, but also won't require your logout link to be a    
  delete "/logout", AuthenticationController, :logout

  scope "/auth" do
    get "/:provider", OAuthController, :request, as: :oauth
    get "/:provider/callback", OAuthController, :callback, as: :oauth
  end

  scope "/", do
    pipe_through :authentication_required

    get "/dashboard", DashboardController, :show
  end
end

# We'll cover this next week
scope "/hook", YourAppWeb.Hook do
  pipe_through :api

  post "/github", GithubController, :received, as: :github_webhook
end

Since you've decided to rely purely on OAuth for your registration and login workflows, instead of a UserController and SessionController for managing your users and logged in sessions, you're making an AuthenticationController for the front-facing register and login pages (as well as the logout functionality) and an OAuthController for handling the backend handoff to retrieve your users' credentials from various OAuth providers, as well as logging them in once everything's copacetic. You've also got an :authentication_required pipeline now, which allows you to softly kick users who aren't logged in, out of the parts of your app that they shouldn't access.

Now, obviously, we need to talk about the YourApp.UserAuth module you're importing the :fetch_current_user and :require_signin plugs from, but there's extra functionality in UserAuth whose usages we haven't seen in the workflow quite yet, and since I want to avoid hopping into and out of each module's code multiple times as I describe how they all fit together, it makes more sense to go through the workflow layer by layer⁹ instead of drilling down into deeper modules as soon as we encounter the first usage of a novel function.

Controllers, Views, and templates

So with that in mind, AuthenticationController is the next step on our journey!

defmodule YourAppWeb.AuthenticationController do
  use YourAppWeb, :controller
  alias YourAppWeb.UserAuth

  def register(conn, _) do
    render(conn, "register.html")
  end

  def login(conn, _) do
    render(conn, "login.html")
  end

  def logout(conn, _params) do
    conn
     |> UserAuth.log_out
     |> redirect(to: Routes.root_path(conn, :index))
  end
end

The templates for register.html and login.html are almost identical. Of course, as you're using the Slime template engine, the differences are much easier to spot:

/- lib/your_app_web/templates/authentication/register.html.slime

h1 Sign Up

h3 Choose your repo host

.services
  = link to: app_installation_page(), class: "button github" do
      img.icon src=Routes.static_path(@conn, "/images/octoutline.svg") alt="Github Logo"
      ' Connect through Github

p Already registered? #{link "Log in instead", to: Routes.authentication_path(@conn, :login)}.

/- lib/your_app_web/templates/authentication/login.html.slime

h1 Log In

h3 using the host you signed up with

.services
  = link to: Routes.oauth_path(@conn, :request, :github), class: "button github" do
    img.icon src=Routes.static_path(@conn, "/images/octoutline.svg") alt="Github Logo"
    ' Github

p Not a user yet? #{link "Sign Up instead", to: Routes.authentication_path(@conn, :register)}.

Minor differences in copy aside, the thing to pay special attention to is the URL your app uses to send users to Github's site. The login page uses the normal link to: Routes.some_derived_path(...), ... do function that you'll use regularly in your templates to allow users to reach various parts of your app. In this case, it's to the oauth_path for the request action for the provider github, which (you'll soon find) provides you literally no direct insight into how your users get from there to Github's site. In contrast, the register page uses app_installation_page/0, which you've defined as a helper within YourAppWeb.AuthenticationView:

defmodule YourAppWeb.AuthenticationView do
  use YourAppWeb, :view

  @app_installation_page "https://github.com/apps/"
    <> Application.get_env(:your_app, :github)[:app_name]
    <> "/installations/new"

  def app_installation_page, do: @app_installation_page
end

A note on module attributes
This pattern may seem familiar to people coming from a background working with Ruby and Rails, where a declaration like @app_installation_page would signify that you were creating an instance variable. And in a few ways, module attributes can behave somewhat similarly to instance variables... just, in a very "immutable functional programming" kind of way. Also, not at all.

The important thing to note in this usage is that module attributes are finalized at compile time, meaning the value of @app_installation_page will be inlined at its call sites during your app's build phase, and even if the result of Application.get_env(:your_app, :github)[:app_name] changes later on, the return of app_installation_page/0 will stay the value that was set according to your environment config.

# config/dev.exs

config :your_app, :github,
  app_name: "something"

# ---

# config/{runtime,releases}.exs

config :your_app, :github,
  app_name: "or_another"

$ iex -S mix

iex> Application.get_env(:your_app, :github)[:app_name]
"or_another"

iex> YourAppWeb.AuthenticationView.app_installation_page
"https://github.com/apps/something/installations/new"

As indicated by the /installations/new portion of the URL, this link will take new users to Github's "Install Github App for the logged in Github User/Organization" page, which is the one that enables all of those permissions you figured out your app needed, whereas the Routes.oauth_path/3 link before the app is installed will take your new users to Github's "Authorize OAuth App for the logged in Github User". The difference is subtle in appearance, but it determines whether your app has the fine-grained permissions it needs to function properly (Github App), or the side-of-a-barn broad category permissions it needs in order to log the user in, and then... do literally nothing else involving an integration with GitHub (OAuth App).

Speaking of OAuth, that's exactly the Controller that makes up the next part of our journey! And, uh... this is a bigger one.

defmodule YourAppWeb.OAuthController do
  use YourAppWeb, :controller

  alias YourApp.Accounts
  alias Accounts.User
  alias YourAppWeb.UserAuth

  plug :set_unique_state when action == :request
  plug :verify_unique_state when action == :callback
  plug Ueberauth

  def callback(%{assigns: %{ueberauth_auth: auth}} = conn, _params) do
    case Accounts.get_or_create_user_with_credentials(auth) do
      user = %User{} ->
        some_success_path = nil # you'll want to fill this in with
                                # something, likely from `Routes`
        conn
         |> UserAuth.log_in(user)
         |> redirect(to: some_success_path)
      %Ecto.Changeset{} ->
        conn
         |> put_flash(
              :error, 
              "Inserting the user and/or some of its associated #{
              }records failed. Check to make sure the schema #{
              }constraints and incoming data all conform."
            )
         |> redirect(to: Routes.authentication_path(conn, :register))
      nil ->
        conn
         |> put_flash(
              :error, 
              "Something went really wrong."
            )                   # and you'll definitely want to debug this.
         |> redirect(to: Routes.authentication_path(conn, :register))
    end
  end
  def callback(conn, %{"error" => error, "error_description" => message}) do
    require Logger
    Logger.error(%{"error" => error, "message" => message})
    conn
     |> put_flash(:error, "OAuth request failed in transit. This has been logged and will be looked into.")
     |> redirect(to: Routes.authentication_path(conn, :register))
  end

  defp set_unique_state(%{query_params: %{"state" => _}} = conn, _), do: conn
  defp set_unique_state(%{path_params: %{"provider" => provider}} = conn, _) do
    state = :crypto.strong_rand_bytes(24) |> Base.url_encode64()

    conn
     |> put_session("oauth_state", state)
     |> redirect(to: Routes.oauth_path(conn, :request, provider, state: state))
     |> halt
  end

  defp verify_unique_state(conn, _) do
    state = fetch_query_params(conn).query_params["state"]
    unique_state = get_session(conn, "oauth_state")

    conn
     |> delete_session("oauth_state")
     |> verify_unique_state(unique_state, state)
  end
  defp verify_unique_state(conn, state, state), do: conn
  defp verify_unique_state(conn, _expected, _tampered) do
    %{path_info: ["auth", provider | _]} = conn
    conn
     |> put_flash(
          :error, 
          "Something occurred while trying to authenticate with #{provider}. Please try again"
        )
     |> redirect(to: Routes.authentication_path(conn, :register))
     |> halt()
  end
end

The plugs :set_unique_state and :verify_unique_state are safeguards for preventing MitM attacks. MitM stands for Malcolm in the Middle, a sitcom which follows a kid genius growing up in a wholly dysfunctional family, and which aired on the Fox network from January 2000 until May 2006 to both critical and popular acclaim. It also stands for Meddler in the Middle, which is a case of internet attack where someone intercepts the data over your connection, and then either steals the sensitive details therein, sends malicious data back to you, or does both of these. These functions provide a safeguard. Details in this footnote ==> ¹⁰

In which Ueberauth plays gatekeeper in more ways than one

There's probably a burning question on your mind: "Where the h*ck is request implemented?" First of all, don't cuss; and second, it turns out that plug Ueberauth handles it. However, as I am neither the author of Ueberauth, nor an eldritch sorcerer, I cannot explain the how, what, or why of any of that plug call's inner workings, try though I have, multiple times. For hours.¹¹

Let's move on; the callback action receives a conn that's already had either an Ueberauth.Auth or Ueberauth.Failure struct added to its assigns under either the :ueberauth_auth or :ueberauth_failure key, respectively¹². The Ueberauth.Auth struct contains everything you need in order to create a User and their corresponding authorization tokens within Accounts.get_or_create_user_with_credentials/1.

Explaining how to create Ecto records out of data received from a controller is outside of the scope of this tutorial.¹³ The way I've structured my app, get_or_create_user_with_credentials/1 persists records to three distinct tables in the database: one for the user, one for their OAuth profile, and one for their OAuth tokens (by default, Github uses a combination of short-lived access tokens and longer-lived [but one-time use] refresh tokens).

The call to Accounts.get_or_create_user_with_credentials/1 occurs in a case clause so that we can delegate what to do based on its success or failure. In this three-split path, receiving a nil value from it means something went wrong that we had no idea could happen, an %Ecto.Changeset{} struct implies that there was an issue trying to persist one or more of the records to the database, and being able to match user = %User{} in the topmost case means we successfully came back out of the function with the user and their credentials all persisted to the database. In the former two cases, we present an error to the user (which should be ourselves until we've worked most of the kinks out of this function). In the latter case, we invoke UserAuth.log_in/2 and then redirect to some path the user would logically be directed to next.

At this point, your user has successfully logged in through OAuth, and has a persistent session. The confetti cannons are overjoyed 🎉

You're not done quite yet though

Of course, those all just compose the outer boundary of YourApp; the parts your users will directly engage for access to the things they're trying to do. That is to say, your user can theoretically go through the complete sign-up/sign-in workflow, but you still have a little more business logic to implement deeper down in YourApp.¹⁴

🎵 Going auth the rails phoenix on a crazy cookie train 🎵

Let's look at YourAppWeb.UserAuthnext:

`UserAuth`

defmodule YourAppWeb.UserAuth do
  import Plug.Conn
  alias Phoenix.Controller
  alias YourAppWeb.Router.Helpers, as: Routes
  alias YourApp.Accounts
  alias Accounts.{User, OauthLogin}

  @doc """
  Retrieves `conn.assigns.current_user`, setting it first if necessary.
  """
  def current_user(%{assigns: %{current_user: user?}}), do: user?
  def current_user(conn), do: conn |> fetch_current_user |> current_user

  @doc """
  Store a user's OAuthLogin.id alongside an expiration time within
  the encrypted session cookie.
  """
  @spec log_in(Plug.Conn.t, %User{oauth_logins: [OauthLogin.t]}) :: Plug.Conn.t
  def log_in(conn, %User{oauth_logins: [login]} = _user) do
    conn
     |> put_session("login_token", login.id)
     |> configure_session(renew: true)
  end
  def log_in(conn, _non_user), do: conn

  @doc """
  Clears the session.
  """
  @spec log_out(Plug.Conn.t) :: Plug.Conn.t
  def log_out(conn), do: configure_session(conn, drop: true)

  @doc """
  Redirects to the login page if `conn.assigns[:current_user]` is `nil`.

  Runs `fetch_current_user` if `:current_user` is unassigned.
  """
  def require_signin(conn, opts \\ [])
  def require_signin(%{assigns: %{current_user: %User{}}} = conn, _),
    do: conn
  def require_signin(%{assigns: %{current_user: nil}} = conn, _),
    do: conn
     |> Controller.put_flash(:error, "You need to be logged in to use that")
     |> Controller.redirect(to: Routes.authentication_path(conn, :login))
     |> halt
  def require_signin(conn, _) when not is_map_key(conn.assigns, :current_user),
    do: conn
     |> fetch_current_user
     |> require_signin

  @doc """
  Loads the user record from the token stored in the session and adds it to the
  `conn` assigns.

  If a user record can't be loaded from the token, say, because
  the token is expired or no corresponding record could be found,
  then the session is dropped and the client will be asked to log in
  again the next time they access a path requiring authorization.

  This plug is memoized; re-running it during a request will
  retrieve the previously-returned value, and not further
  alter the `conn` in any way. To re-run the fetch, delete
  the `:current_user` key from the `conn.assigns` map.
  """
  def fetch_current_user(conn, _opts), do: fetch_current_user(conn)
  def fetch_current_user(%{assigns: %{current_user: _}} = conn), do: conn
  def fetch_current_user(conn) do
    login_token = get_session(conn)["login_token"]
    {conn, user?} = case load_user_from_session_token(login_token) do
      {:ok, user} ->
        {conn, user}
      {:no_session_token, nil} ->
        {conn, nil}
      {:token_refresh_denied, user} ->
        message = "There was a problem refreshing your provider's access token, repos may not display the latest data"
        {Controller.put_flash(conn, :error, message), user}
      {_error, nil} ->
        {log_out(conn), nil}
    end

    assign(conn, :current_user, user?)
  end

  defp load_user_from_session_token(login_id) when is_binary(login_id) do
    with %User{} = user <- Accounts.get_user_with_login(login_id),
         {:ok, user} <- maybe_refresh_user_tokens(user) do
      {:ok, user}
    else
      nil -> {:user_not_found, nil}
      {:token_refresh_denied, _user} = result -> result
    end
  end
  defp load_user_from_session_token(nil), do: {:no_session_token, nil}
  defp load_user_from_session_token(_), do: {:user_not_found, nil}

  @hour 60 * 60
  defp maybe_refresh_user_tokens(%User{oauth_tokens: [bearer_token]} = user) do
    hour_from_now = DateTime.add(DateTime.utc_now(), @hour)
    comparison = DateTime.compare(bearer_token.expires_at, hour_from_now)
    with comp when comp in [:lt, :eq] <- comparison,
         {:ok, tokens} <- Accounts.refresh_bearer_token(bearer_token) do
      {:ok, %{ user | oauth_tokens: tokens }}
    else
      :gt = _comparison -> {:ok, user}
      {_error, _old_token} -> {:token_refresh_denied, user}
    end
  end
end

I've taken care to describe how each public function works within its @doc tag, so enough of this should be straightforward, but a quick rundown of the functions shouldn't hurt anybody.¹⁵

`.current_user/1`

As with most lazy television, we draw in the audience by starting our episode in the middle of the action: UserAuth.current_user will retrieve the already-assigned :current_user from the connection's assigns, or it will invoke fetch_current_user/1 to populate the key and then retrieve its value. But why does it call fetch_current_user/1 at all, and how does fetch_current_user/1 work?

*record scratch* *freeze frame*¹⁶ Yeah, that's (a function to retrieve an app's database representation of) me. You're probably wondering, how'd it get into this situation? Well...

*The Who's "Baba O'Riley" seems to begin playing deep within the very fabric of reality.*

`.log_in/2`

*record scratch* THERE'S NO TIME TO EXPLAIN, we have to flashback to the inciting incident, UserAuth.log_in/2! This function is called with a conn and a YourApp.Accounts.User struct, which the function head pattern-matches on when its oauth_logins relation contains exactly one YourApp.Accounts.OauthLogin struct.¹⁷ This login's id will be our session token.¹⁸ Finally, log_in/2 is going to configure_session(conn, renew: true) before passing the conn back to the function that called log_in. The :renew option makes Plug generate a new session ID to give back to the client within our server's response, to protect from session fixation attacks.

`.log_out/2`

log_out/2 is the most straightforward of all the functions in UserAuth. It prevents the session cookie from being sent back to the client at all on this request. The next action the client takes (which, in the case of our AuthenticationController.logout/2 function, is to load the app's root index page), the app will treat them as a visitor until they log back in as a User. And with that exposition out of the way...

`.require_signin/2`

We still haven't learned anything about how fetch_current_user/1 works, but we've got bigger problems to deal with, because require_signin/2 is creating some serious thematic tension in its presentation of a conflicting philosophy to the one set forward by our established narrative (gasp!)! Like some kind of twisted¹⁹ version of current_user/2 from that mirror dimension in Star Trek where everyone has facial hair that makes them hotter but also, evil, require_signin/2 has many similarities to the latter function, but instead of always letting the conn continue through the Plug pipeline, in a very evil way, it doesn't do that necessarily in every circumstance (bigger gasp!)!

THERE'S NO TIME TO EXPLAIN, but let me try anyway. current_user/2 has a "live and let live" approach to things: If it can't find assigns[:current_user], it'll make a call to fetch it, and it's okay with whatever the fetch finds, whether that's an actual %User{} struct, or a nil value. If current_user/2 sees a nil, it's just like "Hey, client, I know how it is. We all feel a little nil from time to time, but it doesn't mean we're bad protocols." Then it lights a joint and listens to Grateful Dead for the rest of the request while the rest of the pipeline is at work. Meanwhile, require_signin/2's philosophy feels closer to the one embraced by the fictional nation setting of the Bleakpunk computer game Papers, Please. But its workflow is only applied to the parts of your app that go through YourAppWeb.Router's :authentication_required pipeline, so rather than being an unwitting stooge of totalitarianism who's just trying to keep his family fed, require_signin/2 is really more like a hyper-vigilant, omnipresent, standing-slightly-too-close-to-you bouncer. It makes sure every client's name is "on the list" every time an action has them come through its pipeline. When a client even so much as refreshes the page they're on, require_signin/2 swings by and double-checks the list to make sure they're still on it, gently heaving them back out to the lobby the instant it can't verify their listworthiness, telling "VIPs only, buddy", before it goes back to powerwalking the perimeter of the club, incessantly checking the IDs of every user either in or approaching the VIP room.

Actually, require_signin/2's not so bad. Imagine if current_user/2 were the bouncer in front of your VIP room; nobody would ever be turned away! Sure, your clients may be a little annoyed if their session ever expires and they have to sign in again, in a circumstance they find "unexpected", but hey, that session expiration is for their benefit. Like I mentioned when we went over configure_session(conn, renew: true); doing what you can to keep attackers from impersonating your users is crucial. If you didn't have safeguards like renewing the session ID and setting expiration times, and an imposter ever managed to get a copy of a user's token, they could impersonate that user for as long as they wanted, and you wouldn't be able to stop them without taking drastic measures, assuming you found out there was an imposter at all. And if I've learned one thing from watching weird TikTok clips of people doing things that are allegedly relevant to the team-based, route-out-imposters video game sensation of 2020, Among Us!²⁰, it's that imposters can outsmart your teammates and destroy your spaceship. And they will. Every. Single. Match.

Imposters wreak havoc on things, both in outer space, and in cyber-space.

That honestly doesn't have much of anything to do with user authentication workflows or stale session attacks. I just thought putting a colorful, funny gif here would break the monotony of this giant wall of text, a bit.

OKAY, now that I've covered current_user/2, log_in/2, log_out/2, require_signin/2, and teased it almost as long as George R.R. Martin teased Daenerys' dragons, we finally arrive at… The moment we flashed back from!

`.fetch_current_user/2`

*record scratch* THERE'S NO TIME TO EXPLAIN; I need to finish explaining all of this module's functions! fetch_current_user/2 is at the crux of your Phoenix app's session persistence capabilities. It gets the value stored in the session map under the key "login_token" and conditionally operates on conn based on the result of load_user_from_session_token/1. If the result is {:ok, user}, there doesn't have to be any further processing on either the conn or the user in this function, aside from adding the user to conn.assigns. Conversely, with {:no_session_token, nil}, we aren't getting any user to assign to the conn, but we know that since it's because no session token was loaded to begin with, we don't have to log the user out or display any kind of error message about what happened. {:token_refresh_denied, user} means that while the integration didn't allow YourApp to refresh the user's access token and the user will be unable to perform any actions against Github's API, they can still interact with the data cached with YourApp, although YourApp will show a notification that their data may not be in sync with what Github has. Any other sort of response from load_user_from_session_token/1 essentially means that the user couldn't or shouldn't be loaded; right now, fetch_current_user/2 regards those cases as instances where the user should be logged out from the session, but you can change that if your use cases call for different behavior.

Now, a funny thing about teaching: It can help reinforce your memory regarding what you know, but it can also lead you to make new discoveries and learn even more. This tutorial was originally written with a version of load_user_from_session_token/1 that involved verifying that the login_token hadn't expired. This was because neither the series of blog posts, nor the section of Programming Phoenix ≥ 1.4 on session persistence, mentioned any sort of built-in mechanism that the server might use to expire session cookies. Indeed, a random dive into Plug.Session's documentation while writing this tutorial lead me to discover a max_age option, which would have been helpful in the time between writing the comparison logic for both expiring the login token and refreshing the bearer token, and realizing that the API was being called to refresh the bearer token for every action while a user was logged in. Mine is apparently the first Phoenix tutorial to recognize the sublime importance of having your users verify who they are every once in a while. It's fine. The information being helpful to me is why I now pass it on to you 💖

The version of load_user_from_session_token/1 presented here is pretty straightforward. I don't particularly feel like I need to walk you through this one.

maybe_refresh_user_tokens/1 matches on the user struct, like log_in/2, but it's interested in the user's :oauth_tokens as opposed to their :oauth_logins. The single entry here is going to be the bearer token — the token they'll use to interact with Github's API. They also have a refresh token, which Accounts.refresh_bearer_token/1 retrieves and uses to request a new value for the bearer token from the Github integration.²¹ Of course, it only needs to do this if the access token is going to expire soon. If it is, it calls to refresh the token, updates the User with the new value, and then sends the {:ok, updated_user} envelope back to its caller with the new token on a successful refresh. If the access token isn't expiring soon, it passes the user back to the caller. unchanged, in the {:ok, user} envelope. Any error results in it sending {:token_refresh_denied, user} back, so that the user can still use YourApp's integral features.

Although it's only 110 lines of code, it's easy to marvel at the little behemoth UserAuth is within YourApp's workflow. But even if it was daunting to consider when you began, you've learned that Phoenix and Plug lay down enough of the infrastructure necessary to assemble it relying on the help (or hindrance) of a larger third-party package (especially one that takes a token standard designed to allow web apps to confidently send messages to other web apps and tries to squeeze it into the already-solved problem space of session management).²²

We aren't out of the woods quite yet, though. YourApp's endpoint and its config still have a few things to go over.

`YourAppWeb.Endpoint`

defmodule YourAppWeb.Endpoint do
  use Phoenix.Endpoint, otp_app: :your_app
  @endpoint_config Application.get_env(:your_app, YourAppWeb.Endpoint)

  # The session will be stored in the cookie and signed,
  # this means its contents can be read but not tampered with.
  @session_options [
    store: :cookie,
    key: "_your_app_key",
    max_age: @endpoint_config[:session_max_age] || 604800,   
    signing_salt: "f+tfIyAdTnKXiTMc",
    encryption_salt: "4/zKvGZ0eXRikpgT+INdEtLw"
  ]

  socket "/socket", YourAppWeb.UserSocket, websocket: true, longpoll: false

  socket "/live", Phoenix.LiveView.Socket, websocket: [
    connect_info: [session: @session_options]
  ]

  # …

  plug Plug.MethodOverride
  plug Plug.Head
  plug Plug.Session, @session_options
  plug YourAppWeb.Router
end

You might be worried about storing these salts in a file that's checked-in to version control. I'm not a security expert, but the common philosophy in that regard among security experts: Don't panic. It is safe to have a salt be a known value. The important thing to keep from leaking is your secret_key_base. Even if an attacker knows your salts, they still won't be able to forge your signature on a cookie, or decrypt the contents of one that they've gotten hold of without YourApp's secret_key_base. Though, if the paranoia overwhelms you, you can always keep your secret values in your shell environment and then retrieve them in your config files, or put them in files that aren't checked-in to source control.²³

# <your_app_root>/.gitignore
# …
/config/secret/

# or possibly
*.secret.* # extension in the middle of a chain
*.secret   # extension at the end of the chain 
# …

`config/*.exs`

Speaking of config files, that brings us to the last piece of this workflow. You'll need to configure Ueberauth by informing it to use the Github strategy when Github is the OAuth provider like so:

# config/config.exs

config :ueberauth, Ueberauth,
  providers: [
    github: {
      Ueberauth.Strategy.Github, [
        allow_private_emails: true
      ]
    }
  ]

And you'll additionally need to provide Ueberauth's Github strategy with the Client ID Github lists for your app, as well as a Client secret, which you can generate for your app by clicking the button marked "Generate a new client secret" within the header of the Client Secrets section of YourApp's developer settings. It'll look something like this:

(kids, don't try this at home) (by which I mean, don't post screenshots of your client secrets)

# config/{dev,staging,prod,releases,runtime}.exs

config :ueberauth, Ueberauth.Strategy.Github.OAuth,
  client_id: "add in secrets",
  client_secret: "add in secrets"

And you'll want to keep at least the client_secret a secret, using whichever secret-storing strategy sublimely suits your style.

---

You made it!²⁴

...FWAHHH, hello, End of the Post! 24 footnotes, 578 lines, 7660 words, and 47823 characters!²⁵ But we got through it together, all of the code examples are contiguous, working pieces of code (most of them... at least...), nobody had to dig through multiple other blog posts in order to get all of the parts put together, and even I learned a few things in the process. I'd say that just about the only thing that'd do better to prove this was a successful tutorial, would be for you to ~~buy me a Ko-fi~~ help keep my phone line connected!

No, seriously. If you appreciated/enjoyed/learned something from/shamelessly ripped code wholesale from this tutorial, and you want to keep seeing more tutorials like them, it's infinitely easier for me to write them when I can pay my electric bill, my internet bill, and my rent! Donating to my Ko-Fi or directly to me on Square Cash not only helps me to be able to keep this blog going, it also helps me do things like make good on the payment schedule I told my phone provider I'd adhere to for paying off my overdue fees!

Also, let me know down in the comments what aspects of these tutorials I could improve in future posts; better reference materials for languages and frameworks lead to better tools built in those languages and frameworks.

Meanwhile, keep a lookout for the next tutorial in this series: Taming Webhooks, while failing to tame the single mutable component you might ever encounter in all of Elixir

No, I didn't go during Pride, and I have no reason to really suspect this was why I wasn't offered a job. It could've been anything; they probably had more than a dozen strong candidates, and I wasn't among the ones who stood out. I consider myself lucky to have even been considered for a role at thoughtbot, and I hope they're going strong through this pandemic. ↩
Again, I have the utmost respsect for the people at thoughtbot and the work they do. I was kind of annoyed at the structure of these blog posts, but their commitment to readable, well-documented, and efficient test-driven code is an admirable rarity in software. If you can't guess just yet, I like my jokes how I like my summers: Dry, abrasive, and long enough to make you wonder if we're doing enough as a species to handle our carbon footprint. (We're not.) ↩
"Wait, does Guardian know I went to Palm Springs?" ↩
There are benefits to having middleware like Guardian handle your incoming and outgoing JWT credentials, if you need to support JWTs. If you do, then go with Guardian, and godspeed. If you're only using them for session management, though, you really don't need, and definitely shouldn't use JWTs ↩
having worked on everything around user authentication for a good solid few days' worth of work ↩
which was spread over a few months as you went through what you are sure a psychiatrist would classify as "just a harmless bit of mild depression" ↩
"who can say?" en Francais. ↩
If you're not a fan of opening a tunnel from the public internet into port 4000 (or 4001) (or whichever crazy port number you Docker kids use) of your dev box, the other option you have here is to only test your integrations on staging and other remote environments. github.com just plain cannot interpret localhost or 127.0.0.1 to mean your local machine. You may be able to configure your router to serve your dev website, if you're especially savvy, but those really are your really limited options for this particular quandary. ↩
breadth-first code analysis! ↩
The :set_unique_state plug creates a random token that it safely adds to the encrypted session and will send within the request to the authentication provider — Github's server, in this case — and which the authentication provider will pass back to us so that we can verify it prior to doing the work of the callback action, in the :verify_unique_state plug. In that function, we retrieve the copy of the state we'd stored in the session, and compare it to the value we get back in the provider's query parameters to us. If there's a discrepancy, we redirect the client and display an error to them; if they match, we continue through to our callback. There's zero potential for an attacker to build rainbow tables against these tokens, so we can compare them using pattern-matching in the function signatures def verify_unique_state(conn, state, state) and def verify_unique_state(conn, _expected, _tampered). To wit, we're telling the Erlang VM (and Elixir compiler), "This function expects to be called with either the same value passed in at two different spots, or with two different values (whose actual data we don't care about aside from the fact that they're not identical)." In security critical parts of your app, you will want to use a constant-time comparison technique, like ↩
If anyone reading this has a better understanding than I do of how Ueberauth conjures this action from the æther, I would love to know. ↩
I hope it's at least a little obvious where these are coming from. ↩
I already have my topic planned for the next part of this series, but if there's enough interest in it, I can write about creating Ecto records after that! ↩
Yes, I'm counting user authorization and session retention as important business logic. Some web apps incorporate neither of these things, yet still provide a revenue stream to their developers. Your app authenticates users and persists a session for them as part of what it helps them do; if that isn't the case, have you read this far into the tutorial strictly for entertainment value?
↩
If anyone is left still feeling confused after reading this (entire post), don't hesitate to ask questions in the comments. Clarifying the problem points will only make this a better tutorial, and above everything else (besides keeping a roof over my head. roofs are above literally everything in software), I want this tutorial series to leave as few loose ends as possible. ↩
This is the longest it has taken in any media, ever, to get to the *record scratch* *freeze frame* flashback. ↩
You may have noticed that the User schema is using an oauth_logins association, plural, yet our pattern match is expecting exactly one login in that list. This is for two reasons. First, our app anticipates supporting various different OAuth providers, and even with several different OAuth providers, one user will still be one User, which will be an important architecture decision when users want to interact with several different OAuth provider APIs at the same time. Second, although we will eventually have multiple logins per user, right now we only support Github, which only provides one login per user, and so if we found any more than one during any particular query, it'd be a pretty big clue that something wasn't quite right with our database. ↩
Normally, it's dangerous to keep a user identifier in a cookie, because cookies are sent to the client in every request, and all of the JavaScript on the page can read the cookie string. We can be confident that nobody will be able to steal the user's login ID thanks to the minimal amount of configuration Phoenix and Plug.Session need in order provide strong encryption for our app's session data. ↩
At least... "twisted" in the context of lazy, reductive tropes we endure constantly in our TV shows. ↩
it's that I do not understand the kids today and am officially an Old. ↩
While it feels natural to just load your refresh token as part of fetch_current_user/2, there really isn't a compelling reason to. Its sole purpose is to update the bearer token, which only needs to happen about once every 8 hours, and loading it within such a common part of your app is unnecessary runaround for the database in the best case, and exposing it to an unnecessary attack surface in the worst. That's why all of the queries in the Accounts context concerning a user's workflow-relevant data are scoped specifically to just the user's bearer token(s), with the exceptions of the aforementioned refresh_bearer_token/1 and functions related to removing a user's records when they request for their account to be deleted, or when they revoke access to one of their OAuth providers. ↩
If you are curious about how JWTs should typically be used, keep an eye out for the next tutorial in this series. We may have built out letting YourApp's users log in through OAuth with their Github profile today, but next is the fun part: Integrating with Github's developer API 😏 ↩
The benefit of going this route is that if you're deploying using Elixir releases, your app won't need those secret files once it's through its build (compile) phase. The values will be baked into YourApp's optimized BEAM bytecode, and you can configure it to never print them to standard out or a log file. Depending on your security hygiene, this can be much safer than setting environment variables on your server and loading them inside of config/runtime.exs. ↩
unless you ripped all of the code from this tutorial and renamed a few modules and/or functions; then I made it ↩
footnote and wc counts are representative of the statistics of this post up to (and excluding) the link to this footnote ↩

Taming Phoenix (Framework)'s Wild West amid the crippling indifference of Lockdown San Francisco

Josh — Wed, 15 Sep 2021 19:54:31 +0000

Skip ahead to the part about software

The pandemic gave all of us trials we never expected to face. For some, going to bed became an uncomfortable, surreal struggle, while others had a hard time leaving bed completely behind once they had to start their day. Everyone got lonelier (in the U.S., at least), and some people confused small pieces of cloth over their mouth for threats to democracy. Then many of those same people threatened democracy. And, of course, all of this happened against the backdrop of a lot of people getting sick and dying, both of which make it really tough to grease the wheels of Industry by throwing ourselves onto the gears¹.

But I was one of the lucky ones. A person who, even before to the pandemic, had weird dreams aplenty, and no reason to put on pants in the morning; who could distinguish between a public health precaution and an attempted violent overthrow of a peacefully-elected government; who had known loneliness ever since childhood and therefore regarded all of this sheltering-in-place as just another snow day. Most importantly, of course, was the fact that I did not succumb to the horrible illness ravaging the planet. So everything was golden and there were no problems.

Or weren't there? As it turns out, having the most significant events of my life in 2019 be that my best friend abandoned me over a misunderstanding on Independence Day; a company whose work I really believed in fired me in my first 10 days there (and a week before my birthday); another company — who made their offer (for more money) before the first job had even made theirs, and were gracious enough to hold the offer open for me — fired me in my first 3 months there (and a week before Christmas)²; and the few spokes of my support system being flung near and far around the world, meant that I spent all of 2020 and a few months of 2021 wrestling with the towering impression that… maybe the people most important to me, and so many colleagues, could so easily discard me because I truly was worthless, and discardable, to anyone and everyone, and that was how I would always be.

It wasn't at all helpful, either — while people would tell me the usual unemployment insurance requirement of looking for work was suspended in light of the fact that, well, nobody could go look for work, and then later that the unemployment office was offering additional financial support to every unemployed person regardless of circumstance — that despite filing for unemployment well into the throes of the pandemic, my benefits were still bound by the unsympathetic, mechanically-administered requirement of, yes, being able to prove I was looking for work, amid shelter-in-place mandates. I have no idea how much unemployment I was even awarded; I only know that I never received any of it. I was disposable, not only to the first real friend I'd made in the five years I'd been living in the Bay Area, and to both of the companies I'd worked for since mid-2018, but to California itself. It felt like I was being told, "You're emotionally devastated, and unsure of whether you even have any redeeming qualities? Well Tough. If you can't muster the energy and nerve to jump through hoops nobody else is being asked to jump through, then maybe you deserve to be going through debilitating emotional devastation."

Being a huge gamer, my first reaction was to bury myself in the quick satisfaction of challenge-reward loops and cleanly-packed, fully resolved internal narratives that only a PS4 and Nintendo Switch could provide³. Being a programmer, my second reaction was to start putting together a business I could build, scale, and maintain by myself.

And having those three goals – short feature turnaround as complexity increases, graceful (and inexpensive) scaling as the user base grows, and low maintenance and upkeep requirements as its usage ramps up – meant that I might not want to remain in my typical, Ruby on Rails, wheelhouse. A maverick doesn't need vibrant communities gathered around tried and true frameworks with incomparable stores of ready-built solutions to the implementation questions I might face.⁴

These goals would actually be best achieved by building my app in Elixir⁵ and the Phoenix Framework!⁶

Or would they actually? As it further turns out, having little to no guidance through the deep processes that are revealed by not-working-in-Rails has given me firsthand appreciation of just how mountainous, jutting, and craggy the terrain really is underneath the leveled-out, perfectly-trimmed lawn that not-not-working-in-Rails provides developers. In my comfy Rails homestead, questions like "How should I integrate with Github?", "What do I do to verify a webhook payload's signature?", and "OAuth?" weren't a matter of digging through reams of esoteric API documentation, and weren't even a matter of digging through one gem's esoteric readme, but were usually a matter of deciding which gem sparkled brightest amid Rubygems.org's gleaming sea of options; each one eager for recognition, praise, and also, possibly, donations in order to help sustain the developer's basic need for survival idk.⁷

But dagnabbit, I've come this far with my ErlangVM-powered, stubborn, sometimes completely inscrutable functional programming workhorse. I'll be darned if I'm gonna let a little wandering through the desert as I'm dying of thirst make me turn around and reach for Ruby's object oriented, rocket-propelled horseless carriage, that's as configurable, simple, and easy to use as a LEGO playset with thorough, highly detailed instructions and examples. LEGO blocks are toys. I'm not building a toy, I'm building a robust, self-healing, high-uptime software application as a service (SAAaS)! User authentication is better when it takes you two weeks to implement!

But since I did have to figure out so much of this process through my own research, struggle, and persistent reiteration, I figure there might be other people out there, foolhardy enough to say "Maybe my next project would be a good fit for Elixir/Phoenix 🤔." You may be right, you beautiful dreamer, and to help you through some of the rough parts you might encounter, I'm gonna go ahead and tell you how I've been forging my trail. If it helps you cut your own path through this rugged, recursive frontier, then I've made a difference.

You can expect this series to receive regular⁸ updates about the snags, slowdowns, and showstoppers I've pushed through on my way to building a SAAaS⁹ on Phoenix in Elixir. That being said, if you felt concerned at all for my livelihood from reading that section I let you skip over, or if you find yourself appreciating the writing style of these tutorials, any amount you could donate to me through Ko-Fi would make you an honest lifesaver. I'm working on launching my business as fast as I can, but if I want to see it become profitable, first I need to be able to keep my head above water as rent, utilities, internet, phone, food costs, unforeseen expenses, the student loans I'm still repaying a decade later, and (last, but only second to rent in being the farthest thing from "least") interest on repayments all threaten to drown me.

In the meanwhile, soon I'll be publishing my first tutorial in this series, "Taming Github OAuth integrations with Überauth and Guardian (but then actually no just Überauth)", and ~~will update this post with a link as soon as~~ that went up fast! Read it here.

Being alive is nice!

I think that's how that expression goes. ↩
eeeeeyup. Not long stories, but ... not for here, not right now. ↩
If you've never played Dragon Quest XI, Persona 5 (base or Royale), Watch Dogs 2, or Assassin's Creed either Odyssey or Valhalla … definitely don't start playing any of them during the long, dark night of your soul. ↩
I would, of course, later come to question and regret my decision to eschew ready-built solutions to the implementation questions I have faced ↩
Elixir is a great language on a great runtime, and once I have a production server up and running, I'm sure I will discover just how great. ↩
Phoenix is similarly a great framework, but as I have learned, and as you will come to find, it suffers from both a substantial gap between features it provides and features you have to find for yourself, as well as a substantial gap between documentation that's quickly referencable and provides the answers you're looking for, and the documentation that it has. That's not to deride Phoenix or its documentation; explaining a piece of software is hard work, and I'm grateful for the literature its maintainers provide about its ins and outs; it's not as comprehensive as the Rails guide, but Rails has also been downloaded over 295 million times (from Rubygems, which is not where Rails was hosted for the first 45 versions it had) to Phoenix's 56 million total downloads. Bear witness to the network effect in action, people. ↩
please donate to me via Ko-fi! ↩
"Regular" in terms of once the data is smoothed out for irregularities, of course ↩
I know it's not the abbreviation I'm just trying to be funny. I'm just trying to be funny, y'all 😢 ↩

Love Potion `number == 9` | How Elixir Completely Changed My Relationship With Testing

Josh — Wed, 21 Aug 2019 01:00:14 +0000

Back in 2012, I interviewed for a company that emphasized TDD as their primary feature development strategy. At that time, I had been professionally developing software for just under a year, was terrified of JavaScript and the ability one had to just do things with it the moment they opened up Developer Tools, had written five test assertions in total – if that – thanks to the very last lab module I had as part of my computer science minor, and definitely completely lied about knowing what TDD was. But I decided I wouldn't bring up how nervous I was about the fact that we were writing test cases before even knowing what the shape of the code would be. I was just going to power through the interview, and do my best to pick up on the patterns as I was exposed to them.

The day after my flight back home, the phone call came through: I'd gotten the job. And I was incredibly grateful that being able to "fake it til you make it" gave me the opportunity to learn from some of the most driven developers, the most diligent testers, the most dedicated support representatives, and the most detail-oriented designers I've worked with in my career.

The job I had working on Pivotal Tracker taught me so many important lessons, about code and about life, but I lament only letting one fall by the wayside: How to do test-driven development. It is a skill that requires constant practice and upkeep, and it's all too easy for people in our profession to forget the importance of keeping our goal in mind, in favor of hitting the ground running and building whatever we can.

That completely changed once I discovered Elixir. The other day, I implemented an answer to a daily coding challenge in the language. I'll copy the code I wrote for that challenge here, so as to keep the focus of this post all actually in this post:

defmodule W do
  @doc ~S"""
    Takes in a string and returns a list of variations of that string, each containing
  one capitalized letter from the string, in sequence, starting from the left.

  ## Examples

      iex>W.ave("hello.")
      ["Hello.", "hEllo.", "heLlo.", "helLo.", "hellO.", "hello."]

      iex>W.ave("Hello World!")
      ["Hello world!", "hEllo world!", "heLlo world!", "helLo world!", "hellO world!",
       "hello World!", "hello wOrld!", "hello woRld!", "hello worLd!", "hello worlD!"]
  """
  def ave(string) do
    string
      |> String.downcase
      |> String.to_charlist
      |> do_wave # private function
  end

  defp do_wave(charlist, waveform \\ [], waved \\ '') # no `do` block because this signature is used to define defaults
  # ...
end

This code takes an input string, downcases it, and then converts it to an older format used by Erlang known as a charlist (which are made using single-quotes), and processes that list to generate the series of strings that were asked for in the challenge description. Now, let's look at the testing code for this module:

# ./test/w_test.exs
defmodule WTest do
  use ExUnit.Case, async: true
  doctest W
end

Considerably less to look at, right? Let's break down what's going on here:

first, I'm declaring the test module name with defmodule. defmodule is kind of synonymous to class declarations in object-oriented languages, except since there are no objects in Elixir, all of the functions and macros you define in a module exist within, and are called directly from, the module itself, like String.length(string); you never call a method directly from any of your variables or data structures in Elixir! Anyway, it's convention to usually name your test modules "Test", to keep everything simple
The next line, use ExUnit.Case, async: true, pulls in code from one of the modules from ExUnit, Elixir's built-in testing framework. use is a keyword (specifically, it's a special form) that tells the compiler to import ExUnit.Case and load its functions and macros so they can be used in this module, as well as to run a block of code that performs some extra functions. We pass async: true so that these tests can run concurrently to ones in other testfiles, which speeds up test runs, and allows for a greater degree of random order of execution
The penultimate line in this file, doctest W, tells the test runner … 🤔

You know what? I've completely gotten ahead of myself 😐 let's run this testfile.

2019-07-07 19:07:07 ⌚ ruby 2.6.3p62 Newtons-Mac in ~/workspace/elixir/joywave
±  |master ✓| → mix test test/w_test.exs 
Compiling 1 file (.ex)
.

  1) doctest W.ave/1 (1) (WTest)
     test/ext/w_test.exs:3
     Doctest failed
     doctest:
       iex>W.ave("Hello.")
       ["Hello.", "hEllo.", "heLlo.", "helLo.", "hellO.", "hello."]
     code:  W.ave("Hello.") === ["Hello.", "hEllo.", "heLlo.", "helLo.", "hellO.", "hello."]
     left:  ["Hello.", "hEllo.", "heLlo.", "helLo.", "hellO."]
     right: ["Hello.", "hEllo.", "heLlo.", "helLo.", "hellO.", "hello."]
     stacktrace:
       lib/joywave/w.ex:8: W (module)

The magic you just witnessed, is called the examples in W.ave/1's documentation of how to invoke it were just run as actual tests. Elixir treats documentation as a first-class citizen of the language, and that's no more apparent than when the exact same usage examples you provide in your function documentation are unit specs that will be run as just another part of your suite.

This fact changed the entire way I've thought about unit specs, about documentation, and about test-driven development as a practice. Frameworks like Minitest and RSpec come with their own domain-specific language, which requires some time to learn and understand in and of themselves.

And yes, some of the more powerful features of those frameworks can't be replaced, but being able to demonstrate to consumers of your library how to use a given function, or a given language paradigm, at the same time that you're demonstrating to yourself that your code does what it's supposed to, eliminates so much friction when it comes to writing tests for your code. It made it actually possible for me to consider what the output of my code should be before thinking of the implementation. It made me care about when my documentation didn't align with the actual behavior of what the code was doing. It made me want to write documentation; it made me want to documentate ALL the APIs.

It made me want to tell spellchecker that it's not queen of the world, criticizing my awesome new word "documentate".

There are practices worth investing time in when it comes to anything you make your living doing. Test suites are essential for catching regressions, and TDD and exploratory testing help you solidify your user experience and harden your code against edge cases, before both become a soupy murk of uncertain call procedures and thickets upon thickets of uncharted behavior, respectively. Documentation may seem like a hassle, but if you ever want engineers to understand how to use your framework, library, or API, you need to be able to tell them how to use your framework, library, or API.

If you find yourself struggling with test-driven development, with testing in general, with writing documentation, or with understanding the nuances of your company's preferred testing framework, see if your language has a package or module for running doctests. If they don't, consider learning Elixir. It's got some incredibly fun features, some of the best reference material I've seen in my life, and it's powered by one of the best runtime environments for concurrent, fault-tolerant programming in the entire industry. On top of that, you'll probably find yourself learning some great patterns for building reliable, more readable code in all of your known languages.

Now go forth, and dev greatly ✨🐶✨

Ruby quick tip: ABC, Always Be Constantizing

Josh — Fri, 16 Aug 2019 17:00:00 +0000

Ruby has a reputation as one of the most fast and loose languages in modern programming, but in reality, it has a number of subtle features that help ensure the intent of your code is fairly clear.

For example, if you try to define a class like this...

# ./puppy.rb
class puppy
end

The interpreter will tell you

SyntaxError (./puppy.rb:2: class/module name must be CONSTANT)

And you will have no puppy to speak of :[

Constants have a lot of great uses, but their biggest strength is in helping you organize your code and provide context around what a literal is used for.

class Action < ApplicationRecord
  scope :accepted, -> { where(type: "accept") }
  scope :rejected, -> { where(type: "reject") }
  scope :requested_changes, -> { where(type: "changes_requested") }

  # ...
end

These scopes contain string literals that likely aren't going to change, but suppose a major rewrite of this app's business logic occurred, and they suddenly did. If a developer misses one of the invocations of that literal – say, in another file like user.rb, you might not find out about it until the exception surfaces in your runtime.

You can prevent that from happening by moving these string literals to constants, and then referencing those in place of the literals themselves:

class Action < ApplicationRecord
  module Type
    ACCEPT = "accept"
    REJECT = "reject"
    REQUEST_CHANGES = "changes_requested"
    MAKE_BETTER = "improvements_required"
    RUBY_PRETTIER = "improve_readability"

    ALL = [ACCEPT, REJECT, REQUEST_CHANGES, MAKE_BETTER, RUBY_PRETTIER]
  end

  scope :accepted, -> { where(type: Type::ACCEPT) }
  scope :rejected, -> { where(type: Type::REJECT) }
  scope :requested_changes, -> { where(type: Type::REQUEST_CHANGES) }
end

You can even declare these constants within the array literal ALL, and have access to the individual strings and collection, all from one terse declaration.

class Action < ApplicationRecord
  module Type
    ALL = [
      ACCEPT = "accept",
      REJECT = "reject",
      REQUEST_CHANGES = "changes_requested",
      MAKE_BETTER = "improvements_required",
      RUBY_PRETTIER = "improve_readability"  
    ]
  end

  # ...
end

In your terminal:

 2019-03-03 15:33:33 ⌚ ruby 2.6.3p62 Newtons-Mac in ~/workspace/my_great_app
±  |master ✓| → rails console
Running via Spring preloader in process 90221
Loading development environment (Rails 5.2.3)
irb(main):001:0> Action::Type::ACCEPT
=> "accept"
irb(main):002:0> Action::Type::ALL
=> ["accept", "reject", "changes_requested", "improvements_required", "improve_readability"]

After that refactor, you'll also get tab completion in your IDE of choice. Bonus. 👍

Now go forth, and dev greatly ✨🐶✨

O(1) Dev Complexity - How refactors keep your app reliable, and your development time to a minimum

Josh — Thu, 15 Aug 2019 19:54:46 +0000

Jump ahead to the refactors

When I get too proud of the industry I've stumbled into, I like to stop and reflect on the fact that software is likely at the heart of a massive displacement crisis in this part of the country; our chief achievements in the past decade haven't been all that remarkable in hindsight; much of software is a ridiculously fraught house of cards we're building around ourselves; and our big drive now is to create money out of thin air (and, you know, vast, unconscionable amounts of the planet's energy supply).

After that grounding exercise, I reflect on what attracted me to programming in the first place: The puzzle-solving it asks of you, the freeform nature and critical thinking involved in finding a solution, the myriad ways it overlaps with rhetoric and writing, and how if you make it your profession, as John put it, you should

[a]bandon the idea that you are ever going to finish.

— Steinbeck: A Life in Letters

One can almost always find a better way to write a paragraph, just as one can almost always find a better way to write a block of code. Part of this comes from the fact that "better" is a subjective term, but the other half of the equation is that putting thoughts into words is a rough art, and our language isn't necessarily as precise – certainly nowhere near as acute – as the feeling we try to convey with it. Such is also the way of processes and workflows; if you've ever felt the joy of programming in Turtle (I haven't), you will understand the paradox of trying to succinctly yet clearly convey your intentions to a well-meaning but all-too-straightforward creature. Conveying intentions to another human being through the spoken word is far more perilous, outside of the scope of this post, and best regarded by the reader as impossible.

But edits to both prose and program are, indeed, achievable, and to borrow another quote,

To write is human, to edit is divine.

— Stephen King, On Writing: A Memoir of the Craft

Much is said about technical debt, and "the business value" it provides. Conventional wisdom says that "if it isn't a feature that the user sees, it provides no benefit to them, and is not worth the investment." By this rationale, we as developers ought to abandon all private methods, 95-98% of our server logic, 100% of our test cases – who's afraid of feature regressions, anyway? – support and QA teams… essentially everything that isn't your business's marketing team, sales team, and anything that isn't eventually rendered within the client interface (also, keep the metrics. Executives. Like. Metrics. Charts are their gods.)

But the truth is that technical debt inevitably manifests in effects that the user will see, and chasing after the business value ignores this fact in pursuit of delivering features, which makes actually delivering features grow in complexity on a nonlinear scale. The more time your team spends figuring out just what your code is trying to do, the less time your team has to build new features on top of that code, and worse, the more likely they are to make mistakes as they build those new features. Those mistakes may cause new features to function improperly, existing features to stop functioning properly, or/and features yet-to-be developed that much more difficult to build. Engaging with technical debt, and editing your code to be more clear, concise, and understandable, is vital to maintaining consistent feature turnaround. It might sound weird, but keeping your code organized will also make working in your codebase a much more enjoyable experience.

So let's practice being divine, and refactor some code together.

When should I refactor?

My rule of thumb for deciding when to refactor boils down to about four key questions:

Does this block have more than two levels of indentation than the rest of the code in the file?
Is there more than three branching conditions, either as a chain of if…else if… else if… elses or switch/case/cond expressions?
Is this block of code working to modify more than one object or model?
Do I need to reread the block, even once, in order to understand what it's doing?

If I answer "yes" to any of these questions, I make either a todo or a ticket in my project management tool, to remind myself come back to the code and clean it up when I've finished the feature I'm working on. If I answer "yes" to more than one, I refactor the code as I'm working on the feature. Some of you might balk at that, but trust me, you don't want to get caught up in a feature-drive, and realize after a month or more that you haven't got a clue what the thing you wrote is supposed to actually be doing.

Let's break these down with some examples.

1. More than two levels of internal indentation

In nearly every language these days, your indentation level denotes that work is being done within some kind of subprocess, whether that's a class declaration, a method definition, an if-else case, a switch-like, a loop, or a block, lambda, or internal anonymous function. For instance, here's a sample block of code modeled after something I came across while I was working at an enterprise company, on a project written in Python:

items_with_classes = []
for item in items:
  for tag in item.tags:
    if tag.type in current_page.selected_types:
      items_with_classes.push({tag, tag.type})

This isn't the exact code, of course. Partly because I'm pretty sure my NDA forbids me from "taking" any code I've seen at that job and using it elsewhere, and partly because it was over a year and a half ago that I encountered this. But this is the same basic structure of that block, and the actual behavior of it was nested three levels deep within the function I found it in. In essentially any language you work with, there will be constructs for handling this situation more elegantly. In most cases, you can extract the behavior of each loop to its own separate function, but Python has an even more elegant way of creating a new list out of extracted data from lists: A comprehension.

items_with_classes = [{tag, tag.type} for item in items for tag in item.tags if tag.type in current_page.selected_types]

Three levels of nested logic just became a single line of code. And, you are still free to break the comprehension out across more than one line, if you don't find this very readable.

items_with_classes = [{tag, tag.type} 
  for item in items 
  for tag in item.tags 
  if tag.type in current_page.selected_types]

In either situation, the intent of the items_with_classes list is now front-loaded with the information about what each item is going to use as its class, and it was done by leveraging one of the more interesting and expressive features of the language. (Plus, it's still one less line of code 😛)

2. More than three cases of branching logic

Applications become increasingly complex as they grow to handle more variations of data, and this becomes a point of developer overhead once the data variations have to be codified and handled in their own specific ways, and the rate at which that overhead grows will depend on how capable your language is at abstractions and dynamic retrieval and operation.

When you're building a front-end using React and the Flux paradigm, it's typical to have a state object at the root of your application, which is passed down through the component tree to the rendered nodes, and which handles storing all the data your app will use to determine what to render, how it's rendered, and when to render it. This example is based on an internal tool I worked on at the same company as the previous example. I strongly suspect the engineer who wrote it came from a Java background… not to say anything negative about Java, it just felt like this pattern adhered to its strict constraints around getters and setters, as opposed to… well, any other language, besides maybe C.

function updateSearchStateAttribute(state, attrName, newValue) {
  var searchAttributes = state.searchAttributes;

  if (attrName === "inputField") {
    searchAttributes.inputField = newValue;
  } else if (attrName === "searchResults") {
    searchAttributes.searchResults = newValue;
  } else if (attrName === "mediaType") {
    searchAttributes.mediaType = newValue;
  } else if (/*…*/) {
    /* four cases later... */
  } else {
    throw new Error("attribute " + attrName + " does not exist");
  }

  state.searchAttributes = searchAttributes;
  return state;
}

I had no words. And the application had Babel and Webpack. And I still have no words, but I wound up refactoring about two dozen functions that mimicked this pattern, with varying numbers of else if cases, down to this:

function updateStateAttribute(state, sectionName, attrName, value) {
  const section = state[sectionName];
  if (!section) {
    throw new Error(`${sectionName} is not part of the state object.`);
  } else if (!section.hasOwnProperty(attrName)) {
    throw new Error(`${attrName} is not part of ${sectionName} state.`);
  }
  section[attrName] = value;
  return {
    ...state,
    section
  }
}

Again, the language constructs offered an elegant means of removing mental overhead, and this time, removing such a substantial amount of conditional logic not only meant the amount of mental overhead for reading the code improved, but that writing new code required less mental overhead, which substantially improved feature turnaround on this application.

3. More than one kind of entity being handled in one block

There can sometimes be exceptions to this point, since models will eventually have to interact with each other to meaningfully relate in the context of what the user is doing, but as a general rule of thumb, if you find one method taking on more than one responsibility, you should consider if there's some way you could refactor it.

This example is one of my own, and although it has no state-machine components itself, it's inspired by a pattern I refactored within one company's custom-built state machine. Let's suppose I'm building a class to codify my own personal workflows, and I'm implementing the method for how I make_pets_happy…

class Me < Person
  # def initialize ... end

  def make_pets_happy(pets = self.pets) # mebbe I'm takin' care of other ppls pets idk
    pets.each do |pet|
      water = Water.new # if only creating water were this easy in real life
      case pet.class.to_s
      when "Dog"
        food = DogFood.new                   # always dogfood your code, if you can
        bed = pet.bed.good_condition? ? pet.bed : DogBed.new
        toy = BALLBALLBALLBALLBALL.new      # OMG A BALL
        rawhide = Rawhide.new
        pet.feed(food)
        pet.play(toy)
        pet.fill_water(water)
        pet.reward(rawhide) if pet.goodgirl? # always returns `true` for `Dog`
                                             # aliased to `#goodboy?` on `Pet` and `#gooddog?` on `Dog`
        pet.sleep(bed) if pet.tired?
      when "Cat"
        food = FancyFeast.new
        bed = pet.caretaker.bed
        self.bed = CatBed.new # I... I guess I can sleep here...
        toy = Catnip.new
        world = $world        # grab global `world`
        pet.feed(food)
        pet.play(toy)
        pet.fill_water(water)
        pet.reward(world)  if pet.goodgirl? # returns `true` while cat thinks it's actually *your* owner.
        pet.sleep(bed) if pet.tired?
      when "Bird"
        food = BirdSeed.new
        bed = nil # birds sleep standing up, nerd.
        toy = WhateverBirdsPlayWith.new # I dunno        
        pet.feed(food)
        pet.play(toy)
        pet.fill_water(water)
        pet.sleep(bed) if pet.tired?
      end
    end
  end
end

Now, it probably just looks like a really big method to the untrained eye, but in reality, this is an unmitigated disaster of maintainability. Suppose there needed to be an update to this behavior tree, and I suddenly had to worry about my budget, which were defined as budget = self.wallet.funds. The code is all imperative, hard-coded line by line, so in order to take the new budget variable into account, I would need to make sure I found every line initializing a new object, and either add another line beneath it to take budget -= whatever_object_was_initialized.price, or pass budget as an argument to ::new. If I missed an invocation, I would either be miscounting the amount of money I'm spending each time I initialized that object, or getting charged with a misdemeanor for taking something from the store without paying for it (aliased to #shoplifting).

Moreover, what if I start taking care of Hamsters, Lemurs, or Cuttlefish? I would have to add another block of code to this case expression, making the entire coditional that much more brittle in the process. The way to rein in this chaos is something most computer science courses don't teach until near the end of the semester: Polymorphism.

def make_pets_happy(pets = self.pets)
  pets.each { |pet| pet.fulfill(self.wallet.funds) }
end

That's all this method needs to be. One line where I pass my budget constraint in to a method, defined on each class of pet that I might take care of, where the individual branches of the case up above is contained and abstracted away from Me. Me need not worry about any of that. Me should only worry about Me. … … … Not … not "me" me, I should worry about the world around me… You get what I mean, don't pretend that you don't.¹

4. "Wait, what did I just read?"

I'll make this quick, because this post is starting to get pretty long… The last common refactor I worry about is how easy it is to parse what an expression or statement is actually doing. I don't know about you, but long Boolean expressions and inverted Boolean logic are two of the most painful banes of my existence as a software developer.

Let's imagine a #valid? method on some ActiveRecord model:

def valid? 
  state.in?(['active', 'pending', 'finished', 'drafted', 'published', 'deferred', 'unrequited'])
    && user_id.present? && user_id < 5000
    || state.in?(['finished', 'drafted', 'published'])
    && data['status'].between?(200, 299)
end

If I stared at this long enough, I could probably surmise that it was some kind of HTTP request, and #valid? was trying to make sure it was either being performed by a specific user (likely an admin), and in a recognized state, or that it was a request being made by a normal user and in a complete-like state, and that the request status came back within the HTTP success range of statuses. That's all stuff I could codify as discrete methods that the resolved Boolean of parts of this expression. I should also codify the literals in here as constants on their respective classes

COMPLETED_STATES = %w[finished drafted published]
RECOGNIZED_STATES = COMPLETED_STATES + %w[active pending deferred unrequited]

def valid?
  admin_request_in_recognized_state?
    || successful_complete_request?
end

def admin_request_in_recognized_state?
  @state.in?(RECOGNIZED_STATES) 
    && @user_id.in?(User::ADMIN_IDS)
end

def successful_complete_request?
  @state.in?(COMPLETED_STATES) && JSON.parse(@data)['status'].success?
end

Conclusion, finally.

Refactoring is more than just about code clarity and readability. At WWDC 2018, Dave Abrahams gave a talk about how abstracting away from loops and toward algorithms improved not only code readability, but code performance as well (if the video doesn't play, it might only be playable in Safari... let me know if that's the case, and I'll do everything I can to get it into a more universal codec). You can't work toward faster coding implementation patterns, until you begin thinking in terms of abstracted coding patterns, and in order to abstract most of the code in your codebase, you have to begin thinking in terms of how to refactor your code.

To code is human; to refactor? Well... you know the rest. 😉

Thanks for taking the time to read all of this!

but if you don't get what I mean then please ask in the comments and I'll be more than happy to describe the logic behind Me. ↩

DEV Community: Josh

Taming Github OAuth integrations in Phoenix with the help of Ueberauth and Guardian (but then actually no just Ueberauth)

The problem comes into focus

Caturday afternoon accomplishments

This tutorial: The Code Parts

Dependencies

Router

Controllers, Views, and templates

In which Ueberauth plays gatekeeper in more ways than one

You're not done quite yet though

🎵 Going auth the rails phoenix on a crazy cookie train 🎵

`UserAuth`

`.current_user/1`

`.log_in/2`

`.log_out/2`

`.require_signin/2`

`.fetch_current_user/2`

`YourAppWeb.Endpoint`

`config/*.exs`

You made it!²⁴

Taming Phoenix (Framework)'s Wild West amid the crippling indifference of Lockdown San Francisco

Love Potion `number == 9` | How Elixir Completely Changed My Relationship With Testing

It made me want to tell spellchecker that it's not queen of the world, criticizing my awesome new word "documentate".

Ruby quick tip: ABC, Always Be Constantizing

O(1) Dev Complexity - How refactors keep your app reliable, and your development time to a minimum

Jump ahead to the refactors

— Steinbeck: A Life in Letters

— Stephen King, On Writing: A Memoir of the Craft

When should I refactor?

1. More than two levels of internal indentation

2. More than three cases of branching logic

3. More than one kind of entity being handled in one block

4. "Wait, what did I just read?"

Conclusion, finally.

DEV Community: Josh

Taming Github OAuth integrations in Phoenix with the help of Ueberauth and Guardian (but then actually no just Ueberauth)

The problem comes into focus

Caturday afternoon accomplishments

This tutorial: The Code Parts

Dependencies

Router

Controllers, Views, and templates

In which Ueberauth plays gatekeeper in more ways than one

You're not done quite yet though

🎵 Going auth the rails phoenix on a crazy cookie train 🎵

UserAuth

.current_user/1

.log_in/2

.log_out/2

.require_signin/2

.fetch_current_user/2

YourAppWeb.Endpoint

config/*.exs

You made it!24

Taming Phoenix (Framework)'s Wild West amid the crippling indifference of Lockdown San Francisco

Love Potion `number == 9` | How Elixir Completely Changed My Relationship With Testing

It made me want to tell spellchecker that it's not queen of the world, criticizing my awesome new word "documentate".

Ruby quick tip: ABC, Always Be Constantizing

O(1) Dev Complexity - How refactors keep your app reliable, and your development time to a minimum

Jump ahead to the refactors

— Steinbeck: A Life in Letters

— Stephen King, On Writing: A Memoir of the Craft

When should I refactor?

1. More than two levels of internal indentation

2. More than three cases of branching logic

3. More than one kind of entity being handled in one block

4. "Wait, what did I just read?"

Conclusion, finally.

`UserAuth`

`.current_user/1`

`.log_in/2`

`.log_out/2`

`.require_signin/2`

`.fetch_current_user/2`

`YourAppWeb.Endpoint`

`config/*.exs`

You made it!²⁴