DEV Community: Nico Domino

How low-level API calls can stabilize your end-to-end tests

Nico Domino — Tue, 21 Jun 2022 14:57:44 +0000

We're heavy end-to-end monitoring users here at Checkly and always experiment with how to architect our tests the best way. Over the past months, we've settled on a few workflows that make it much easier to spin up new tests, avoid code duplication, and make the entire test setup easier to manage.

One of those strategies is to strictly separate concerns in our tests.

For example, instead of having one check that creates and deletes a resource in the UI, we've split these actions into multiple test runs. One browser check tests the resource creation, and another one the resource deletion. But then, if one test is restricted to resource creation, when do you delete and clean up the resource? And also, if a test only checks the deletion, where do you create the resource in the first place?

This situation is when working on the API level helps. We introduced a custom API client to prepare and tear down all the UI test cases using the underlying API. A few lines of code speeded up the process and made tests more reliable.

We successfully use API calls in our UI tests to execute any prerequisite and postrequisite actions. Below we'll go into more detail on how we split up a large Playwright test codebase and how an API client helped us get there!

Sounds intriguing? Read on!

Plain API calls in end-to-end browser tests

Previously, our Playwright UI tests would create a Checkly dashboard and delete it all in one browser session. This structure led to issues where errors in the "create" phase would immediately cause the test to fail and not execute the test's "delete" portion. We littered our test account with dummy dashboards. To solve this problem, we split our "create" and "delete" tests into separate check runs.

But by decoupling these functionalities, we couldn't rely on a particular setup or clean-up while running the tests. For example, the "delete" test won't garbage collect the "create" test, and the "create" test won't generate entities for the "delete" test to remove.

What if we could work on the API directly to ensure our browser checks are running correctly without leaving traces behind? We introduced a custom API client to stabilize our Playwright tests.

Below is an excerpt of our "Delete Dashboard'' check. The test imports checklyApi from our simpleChecklyApiClient snippet and uses the API client to create a Checkly dashboard so that we can test its deletion in the UI.

This way, the check focuses on one functionality in a nice and clean way!

// DELETE DASHBOARD
const { chromium } = require("playwright")
const { checklyApi } = require("./snippets/simpleChecklyApiClient")

let dashboardId

;(async () => {
  try {
    // Create a dashboard via API, to be deleted right after
    dashboardId = await checklyApi.dashboards.create({
      customUrl: "<https://customdashboard.example.com>",
      header: "Dashboard Creation E2E Test",
    })

    const browser = await chromium.launch()
    const page = await browser.newPage()

    // Try to delete the dashboard using Playwright within a browser session
    // ...
    // ...
  } catch {
    console.error("Error deleting dashboard")
  } finally {
    // If anything fails in that process,
    // we explicitly delete the dashboard to ensure
    // this Check has no left-over side effects
    if (dashboardId) {
      checklyApi.dashboards.remove(dashboardId)
    }
  }
})()

But what happens if the end-to-end test fails to delete the dashboard?

We introduced a finally block to ensure that even if something goes wrong, we'll always clean up the test run by attempting to delete the dashboard with our API client.

A custom Node.js API wrapper

The checklyApi is a JavaScript class wrapper around axios. It includes methods for HTTP verbs that are preconfigured with our baseUrl and an API key. The object also comes with additional convenience methods that target explicit resources in our API (i.e. /checks, /check-groups, etc.). The client is easy to use and tailored to our test scenarios.

// CHECKLY API HELPER
const axios = require("axios")

const API_ROOT = process.env.E2E_API_ROOT_PRODUCTION
const API_TOKEN = process.env.E2E_CHECKLY_API_KEY_PRODUCTION
const ACCOUNT_ID = process.env.E2E_CHECKLY_ACCOUNT_ID_PRODUCTION

const headers = {
  Authorization: `Bearer ${API_TOKEN}`,
  "X-Checkly-Account": ACCOUNT_ID,
}

class ChecklyApi {
  #buildPath(url) {
    return `${API_ROOT}${url}`
  }

  #requestMethodWithNoData(method, url, config) {
    return axios({
      headers,
      ...config,
      method,
      url: this.#buildPath(url),
    })
  }

  #requestMethodWithData(method, url, data, config) {
    return axios({
      headers,
      ...config,
      method,
      url: this.#buildPath(url),
      data,
    })
  }

  async $get(url, config) {
    return (await this.#requestMethodWithNoData("get", url, config)).data
  }

  async $post(url, data, config) {
    return (await this.#requestMethodWithData("post", url, data, config)).data
  }

  // More convenience HTTP Methods

  dashboards = {
    create: (payload) => {
      return this.$post(`/v1/dashboards`, payload)
    },
    remove: (dashboardId) => {
      return this.$delete(`/v1/dashboards/${dashboardId}`)
    },
  }
}

module.exports = {
  checklyApi: new ChecklyApi(),
}

Having all the API client logic centralized in this helper class allows us to make adjustments in a single place and reuse the client across all of our checks. Like in the "Delete Dashboard" example check above, the usage is as simple as:

checklyApi.dashboards.remove(dashboardId)

And for completeness, here is the opposite check "Create Dashboard":

// CREATE DASHBOARD
const { chromium } = require("playwright")
const { checklyApi } = require("./snippets/simpleChecklyApiClient")

const dashboardUrl = "https://testdashboard.example.com"

;(async () => {
  let dashboardIdToDelete

  try {
    const browser = await chromium.launch()
    const page = await browser.newPage()

    // Navigate the UI and try to create a dashboard
    // ...
    // ...

    // Retrieve check ID from the creation response triggered by UI interactions
    const { dashboardId } = await (
      await page.waitForResponse((response) => {
        return (
          response.url().endsWith("/dashboards") &&
          response.request().method() === "POST" &&
          response.status() === 200
        )
      })
    ).json()
    dashboardIdToDelete = dashboardId

    // Dashboard should be visible in the list
    await page.locator(`a[href="${dashboardUrl}"]`).waitFor()
  } catch (e) {
    console.error(e)
  } finally {
    if (dashboardIdToDelete) {
      // Housekeeping, delete the created dashboard via API
      checklyApi.dashboards.remove(dashboardId)
    }
  }
})()

Both Playwright tests leverage plain API calls to set up and tear down, but there's a crucial difference.

How to extract resource ids using Playwright request interception

Combining API calls with end-to-end testing can sometimes be challenging. Setup steps are generally easier to realize than teardowns. But why's that?

When you create resources on the API level, you're in control of the data, requests and responses. But if you want to clean up and delete resources created in a browser session, this data might not be available to you.

This is when Playwright request interceptors shine!

// Retrieve check ID from the creation response
const { dashboardId } = await (
  await page.waitForResponse((response) => {
    return (
      response.url().endsWith("/dashboards") &&
      response.request().method() === "POST" &&
      response.status() === 200
    )
  })
).json()

We run our UI tests as usual and listen to our app requests using waitForResponse to evaluate what resources need to be deleted in the teardown phase. By monitoring the network activity, we can snatch all the required resource ids and delete anything created in your Playwright test using the API.

There are no traces, no matter if the UI test fails or succeeds!

Wrapping Up

Writing fast and stable end-to-end tests can be a real challenge, but keeping tests lean and separating them into smaller chunks helps us to keep our test coverage high without slowing us down.

Our tests are now easier to debug, they share more code, and everything runs faster and in parallel. And even though making API calls in our end-to-end tests felt odd initially, we're not creating sequential test waterfalls anymore, thanks to a tiny Node.js API wrapper. Most of our Playwright tests are idempotent, so that we can run them continuously. And the best part: we're sure everything's cleaned up, no matter how many times we run our tests. This is how it should be!

This practice is only the tip of the iceberg. We're continuously improving our monitoring setup and will share more tricks here on the blog. If you want to learn more about end-to-end monitoring, we've plenty of other resources available. Until then, happy testing!

Adding Authentication to an existing serverless Next.js App in no time with NextAuth.js!

Nico Domino — Sat, 24 Oct 2020 23:08:25 +0000

Updated 25.05.21 with latest NextAuth.js examples / best practices / links

Do you have a Next.js application and you'd like to add user / authentication support? Well look no further! All you need is NextAuth.js, some OAuth keys from your favorite provider (Google, Github, etc.) and about 5 minutes of your time!

Let's summarize what we're going to do here before we get started.

We're going to do the following things in this tutorial:

Create OAuth keys in the Google Developer Portal
Install and configure NextAuth.js into our Next.js application
Protect some page routes and api routes

Let's get started!

Prerequisites

I'm going to assume you have the following already:

A Google Account
An existing Next.js app

Google OAuth Setup

So first we have to obtain our OAuth clientID and secret from Google. NextAuth.js has built-in support for a ton of OAuth providers like Apple, Discord, Facebook, Github, Google, LinkedIn, Okta, Slack, Twitch, Twitter, and a ton more which you can find in their docs - providers. There are instructions on how to retrieve the required OAuth keys for all providers there, but I will only go into detail for how to setup Google here.

First, visit https://console.developers.google.com/apis/credentials.

If you don't have a project setup in Google's developer console yet or you want to create a new one for this application, go ahead and select the project overview in the top left, and click "Create Project"

Once you have the project created, make sure you're in the "API's & Services" -> "Credentials" section of the left-hand main menu. There, select "Create Credentials" and select "OAuth client ID".

If you've not yet gotten the OAuth consent screen setup for your domain, you'll have to do that first - just enter a name for the app and the domain you'll be using for it. For Internal applications there is no Google verification process necessary, but it will also only allow members of your G Suite domain to use the application. Public applications must have the domain verified by Google before large scale sign-in is allowed - OAuth is limited to 100 "sensitive scope logins" i.e. test logins until the consent screen is verified by Google.

Next, select "Web Application" for the type of OAuth Client ID we'd like to create, and give it a name. You'll also have to enter a URI for Authorized Javascript Origins and Authorized redirect URIs here.

The Authorized Javascript Origin is simply the domain of your application, for example myapp.domain.com.

The Authorized redirect URI will vary depending on which provider you're setting up, but for NextAuth.js's default setup with Google as your OAuth provider, the redirect URI will be something like: https://myapp.domain.com/api/auth/callback/google. Obviously make sure to adjust the domain to whatever you're running your application under, the important part is the /api/auth/callback/google path.

After you press "Create", you'll get your OAuth Client ID and OAuth Secret! Make sure to save these somewhere because we'll need them later!

NextAuth.js Install

Next, we're going to need to install NextAuth.js into our Next.js application. So for that, navigate to your project directory and install next-auth.

cd /opt/dev/my-nextjs-app
npm i -s next-auth

Now we're ready for the NextAuth.js setup!

Initial Setup

Now that we've got all the prerequisites and dependencies out of the way, lets get the config file for NextAuth.js setup. First, we'll need a .env file in the root of your project directory if you don't have one yet.

Your .env file should contain the following:

NEXTAUTH_URL=https://myapp.domain.com
GOOGLE_ID=abc123
GOOGLE_SECRET=abc123

The NEXTAUTH_URL is the URL at which your application will be running. You can set multiple URLs for different environments by setting up multiple .env files, like .env.production, .env.development, etc. The same is possible in almost any hosting provider's environment variables setup section (i.e. with Netlify or Vercel, etc.). The GOOGLE_ID and GOOGLE_SECRET are our OAuth keys we received from the Google developer console earlier!

Next, create a Next.js API route for authentication at pages/api/auth/[...nextauth].js. This file should contain at least the following:

import NextAuth from 'next-auth'
import Providers from 'next-auth/providers'

const options = {
  providers: [
    Providers.Google({
      clientId: process.env.GOOGLE_ID,
      clientSecret: process.env.GOOGLE_SECRET
    }),
  ],
  debug: false
}

export default (req, res) => NextAuth(req, res, options)

There's a lot more you can configure here, but this is the minimum requirements for JWT based OAuth authentication. More config options can be found in the docs - configuration, as well as a fully annotated version of this file here.

The keen eyed among you will notice we did not specify any sort of database here. This means a few things:

JWT Support is enabled by default for session handling
We do not have a permanent database of users

If you'd like to permanently store the users/accounts in a database, NextAuth.js makes it really easy to add sqlite for testing or various other adapters such as MySQL, Postgres, MongoDB (via TypeORM), FaunaDB, Firebase, DynamoDB, and Prisma. More information on database details can be found in their docs - adapters.

Next, you'll also want to wrap your app in a NextAuth provider, so open up your pages/_app.js or create one if you don't have one yet.

It should end up looking something like this:

import { Provider } from 'next-auth/client'

export default function App ({ Component, pageProps }) {
  return (
    <Provider session={pageProps.session}>
      <Component {...pageProps} />
    </Provider>
  )
}

By wrapping your whole application in this provider, we can use React Context to make a useSession hook available in any page / component in your app and share session state.

Believe it or not, this is all that's needed for a basic Google OAuth setup with NextAuth.js!

The library includes some basic Login / Logout pages by default, so there's no need to write them. If you'd like to customize those, that is of course supported. Please take a look here for more details.

The last thing we're going to do in this tutorial is to show how you can actually restrict access to pages to authenticated users only.

Currently users can still navigate to any route in our app and will get everything rendered no matter their logged in status because we're not conditionally rendering anything yet.

Restricting Page Access

First, let's build a little AccessDenied component which we'll show the user if they land on a protected route and have not logged in yet. This will tell them they do not have access, and give them an opportunity to sign-in.

Lets name this, components/accessDenied.js

import { signIn } from 'next-auth/client'

export default function AccessDenied () {
  return (
    <>
      <h1>Access Denied</h1>
      <p>
        <a href="/api/auth/signin"
           onClick={(e) => {
           e.preventDefault()
           signIn()
        }}>You must be signed in to view this page</a>
      </p>
    </>
  )
}

Finally, on our pages that we want to protect, we just need to import the useSession hook from next-auth and check for the session before returning the page contents, showing the accessDenied.js component if there is no session available.

Client-side Page

An example protected client-side rendered page (i.e. pages/index.js) may look like this:

import React from 'react'
import { useSession } from 'next-auth/client'
import Layout from '../components/layout'
import AccessDenied from '../components/access-denied'

export default function Page () {
  const [ session, loading ] = useSession()

  // When rendering client side don't display anything until loading is complete
  if (typeof window !== 'undefined' && loading) return null

  // If no session exists, display access denied message
  if (!session) { return  <Layout><AccessDenied/></Layout> }

  // If session exists, display content
  return (
    <Layout>
      <h1>Protected Page</h1>
      <p><strong>Welcome {session.user.name}</strong></p>
    </Layout>
  )
}

SSR Page

And an example SSR page may look like this:

import { getSession } from 'next-auth/client'
import Layout from '../components/layout'
import AccessDenied from '../components/access-denied'

export default function Page ({ session }) {
  // If no session exists, display access denied message
  if (!session) { return  <Layout><AccessDenied/></Layout> }

  // If session exists, display content
  return (
    <Layout>
      <h1>Protected Page</h1>
      <p><strong>Welcome {session.user.name}</strong></p>
    </Layout>
  )
}

export async function getServerSideProps(context) {
  const session = await getSession(context)

  return {
    props: {
      session
    }
  }
}

API Route

And finally, an example protected API Route may look like this:

// pages/api/protected.js
import { getSession } from 'next-auth/client'

export default async (req, res) => {
  const session = await getSession({ req })

  if (session) {
    res.send({ content: 'This is protected content. You can access this content because you are signed in.' })
  } else {
    res.send({ error: 'You must be signed in to access this api route' })
  }
}

Conclusion

Alright everyone, I hope that didn't take much longer than the promised 5 minutes!

This example can be deployed to Vercel for example, as NextAuth.js is compatible with their serverless api functions.

I'd also be remiss if I didn't mention that alongside built-in support for the many OAuth providers I mentioned in the introduction, there is also built-in support for Email based "magic link" sign-in as well as options for adding custom OAuth providers and other custom authentication providers, including a great tutorial detailing how to use LDAP.

I want to thank @iaincollins and @balazsorban and all the wonderful contributors to NextAuth.js for their great work! This post was based mostly off the example app and my experience using NextAuth.js in 2-3 Next.js applications as well as contributing to the project over the last few months.

You can find more information in the official NextAuth.js example application here.