DEV Community: Juan Pablo

Extended "run all specs" feature for Cypress 10

Juan Pablo — Fri, 07 Oct 2022 20:14:54 +0000

Required node packages:

As probably all Cypress fans know already, Cypress 10 was shipped without the "run all tests" feature due to "technical limitations" (they are apparently working on a solution, and here is where you can read all about why they removed it, what are the technical limitations, and all the complaints about it too)

I was, as most of us, in desperate need for the ability to run all tests from the test runner interface, and I stumbled upon this great post from Gleb Bahmutov regarding a workaround/hack to be able to do it until Cypress decides what to do.

The solution that Gleb proposes involves manually importing every new spec to the "all.spec.cy/ts", and then run "all.spec.cy/ts" in Cypress (read the post for more details).

So I saw an opportunity to adapt this to my particular need. In my scenario, I had many tests grouped by folder/feature under e2e. And I knew many people were eventually going to be involved in adding tests to each folder, and that at some point someone was going to forget to add the import statement, and we were going to be in trouble. Then I thought "What if... I can create a script that will, everytime Cypress is executed, navigate the e2e folder, find every spec, and then create the all.spec.cy/ts with the corresponding imports?" And then I went even crazyer "What if I can create a "groupspec" for each folder I have, in case I want to run all tests for a given feature"...

And this is what I came up with

Create the "generateGroups.js" file in your root

/* eslint-disable */
const fs = require('fs');
const folders = fs.readdirSync('cypress/e2e', { withFileTypes: true })

/* Create and populate all.cy.ts with all existing tests imports */
writeFile("all")

addAllImports("all")

/* Create and populate a "group" cy.ts file per folder with corresponding imports */
folders.filter(dirent => dirent.isDirectory())
  .map(dirent => dirent.name).forEach(folder => {
    writeFile(folder);
    addGroupImports(folder)
  })

function addGroupImports(spec) {
  readImports(spec, spec);
}

/* Add all imports to all.cy.ts file */
function addAllImports(spec) {
  folders.filter(dirent => dirent.isDirectory())
    .map(dirent => dirent.name).forEach(folder => {
      readImports("all", folder);
    })
}

/* Build the "import" commands and insert it
in each corresponding cy.ts file */

function readImports(spec, folder) {
  let files = fs.readdirSync(`cypress/e2e/${folder}`)
  let newImport = "";
  files.forEach(file => {
    newImport = `import "./${folder}/${file}";\n`;
    appendImport(spec, newImport);
  })
}

/* Insert the import lines in each corresponding file */
function appendImport(fileName, newImport) {
  fs.appendFile(`cypress/e2e/${fileName}.cy.ts`, newImport, function (err) {
    if (err) throw err;
  })
}

/* Create group cy.ts file */
function writeFile(file) {
  fs.writeFile(`cypress/e2e/${file}.cy.ts`, "", function (err) {
    if (err) throw err;
    console.log(`${file} Group Spec file was created successfully.`);
  })
}

And then this would be your scripts section in the package.json file:

"scripts": {
    "open": "npm run groups && cypress open --browser chrome --e2e",
    "test": "npm run cleangroups && cypress run --e2e",           
    "groups": "node generateGroups.js",
    "cleangroups": "npx rimraf cypress/e2e/*.cy.ts",
  },

where:

"open" will create the group files and then open cypress, where you will have a .cy.ts file for each folder, and the all.cy.ts file

"test" will use rimraf (a node package that will allow you to delete files/folder cross platform) to delete all group files, so when you run "cypress run" it will run all tests (if you don't delete the group files, all tests will be executed twice)

"groups" executes generateGroups to create the group cy.ts files

"cleangroups" to delete the files when needed

This is how my folder structure looks normally (without group files)

This is how it looks when I run "npm run open"

And this is Cypress:

This is what my all.cy.ts file looks like:

Again, I love the flexibility I have to adapt stuff to fit my needs... :D

Hope this is useful for anyone... Have a great day and...

Happy testing!

How to test sent and received emails with Cypress 10, Ethereal and Nodemailer

Juan Pablo — Fri, 30 Sep 2022 19:55:34 +0000

Last week I was assigned the task of finding out if Cypress was well suited to test email flows. Keep in mind that for different reasons the Cypress framework I built is not integrated into the app code, and even the application under test can't be deployed locally, so the tests always run against the live app deployed on 3 environments (dev, stage and some tests against prod)

Since I am a Cypress fan my first answer to those assignments/questions is always "of course! Cypress can do anything!"... I must admit that I sometimes regret having said that, not because I found it impossible to do, but because it usually means that I have to spend quite some time researching and building POC's.. but what the heck, that's part of the fun! :D

Anyway... for this particular assignment I quickly found a couple of posts with related information. The first one was "Testing HTML emails using Cypress" by Gleb Bahmutov. But I found a problem. The solution described makes use of a plugin called smtp-tester, which runs a local smtp server, and in order to us it you need to manipulate your app's (the app under test) configuration so that, when executed locally, you can send the emails using the local smtp configuration. For me that was a NO-GO for the reasons I mentiond above.

So I kept digging, also asked some questions on the Cypress' gitter channel (I encourage everyone to subscribe to it), and Gleb directed me on the direction of Ethereal.email. I even found this post by him too "Full Testing of HTML Emails using SendGrid and Ethereal Accounts". I had never heard about it, but it's a really cool tool. It's basically an online service that creates a fake smtp server for you to use, where emails are actually never sent anywhere, and are deleted after a couple of hours.

Then I needed the bit related to actually fetching the emails from the ethereal account, and I found this post which shows how to do it, using the imap-simple node package. But now my new problem was that the particular package seems outdated and not maintained any more, and I needed something more "reliable" (the company does regular security scans on everything used). So I kept searching and found imapflow, which basically does the same thing, but with some little differences.

Here are the two main cases for email testing that we will find:

Signup process email verification - you want to test if the email that the signup process sends to the new user is correct (right format, right content, probably check if the verification code sent actually works, etc)
Email notifications: whatever processes in the app that trigger a notification email.

This example will use the cypress dashboard signup page for testing the flow (you can use the signup page of the app you are trying to test). To understand exactly how ethereal emails work and how they integrate with this email testing pattern I recommend you to visit the links I shared above (there are some nice diagrams there too)

Too much talk (or typing)... let's get to work

Requirements:

Cypress 10.3^
Plugins:
recurse
nodemailer
imapflow
mailparser

Note that a good part of the code I'll share was extracted from Gleb Bahmutov's solution for testing html emails, and also extracted some ideas from this post. All I did was put the pieces together, and give it my own flavor

These are three basic steps: (The code is properly commented to understand exactly what happens at each step)

Create your plugins
Prepare your config file
Write your test

The original solution contains all methods in one plugin, but what I did was separate each method in its own plugin file.

create-account.js
get-last-email.js
parse-email.js
send-email.js

(I'm sure I don't need to explain what each plugin does)

cypress/plugins/create-account.js

const nodemailer = require("nodemailer")

const createAccount = async () => {
    let testAccount
    testAccount = await nodemailer.createTestAccount()
    console.log("created new email account %s", testAccount.user)
    console.log("for debugging, the password is %s", testAccount.pass)

    return testAccount
}

module.exports = createAccount

cypress/plugins/get-last-email.js

// use Nodemailer to get an Ethereal email inbox
// https://nodemailer.com/about/
const nodemailer = require("nodemailer")
// used to fetch emails from the inbox via imap protocol
// https://github.com/postalsys/imapflow
const { ImapFlow } = require("imapflow")

const getLastEmail = async (user, pass) => {
    debugger
    let client = new ImapFlow({
        host: "ethereal.email",
        port: 993,
        secure: true,
        auth: {
            user: user,
            pass: pass
        }
    })
    await client.connect()
    let message
    // Select and lock a mailbox. Throws if mailbox does not exist
    let lock = await client.getMailboxLock("INBOX")
    try {
        message = await client.fetchOne(client.mailbox.exists, { source: true })
        // list subjects for all messages
        // uid value is always included in FETCH response, envelope strings are in unicode.
        // for await (let message of client.fetch("1:*", { envelope: true })) {
        //     console.log(`${message.uid}: ${message.envelope.subject}`)
        // }
    } finally {
        // Make sure lock is released, otherwise next `getMailboxLock()` never returns
        await client.messageFlagsAdd({ seen: false }, ["\\Seen"])
        lock.release()
    }
    await client.logout()

    //If no message was received (message = false) then return message, to ensure that the task will retry
    //If a message is received return the source in order to parse its content
    if (!message)
        return message
    else
        return message.source

}

module.exports = getLastEmail

cypress/plugins/parse-email.js

// used to parse emails from the inbox
const simpleParser = require("mailparser").simpleParser

const parseEmail = async (message) => {
    const source = Buffer.from(message)
    const mail = await simpleParser(
        source
    )

    return {
        subject: mail.subject,
        text: mail.text,
        html: mail.html,
        attachments: mail.attachments
    }
}

module.exports = parseEmail

cypress/plugins/send-email.js

const nodemailer = require("nodemailer")

const sendEmail = async (user, pass, emailObject) => {
    // create reusable transporter object using the default SMTP transport
    let transporter = nodemailer.createTransport({
        host: "smtp.ethereal.email",
        port: 587,
        secure: false, // true for 465, false for other ports
        auth: {
            user: user,
            pass: pass
        },
    })

    let info = await transporter.sendMail(emailObject)

    // Preview only available when sending through an Ethereal account
    console.log("Preview URL: %s", nodemailer.getTestMessageUrl(info))
    return info.messageId
}

module.exports = sendEmail

Config file: we load the plugins, and create the tasks in the setupNodeEvents function that will call the plugin's methods

cypress.config.js

const { defineConfig } = require("cypress");
const getLastEmail = require('./cypress/plugins/get-last-email');
const sendEmail = require('./cypress/plugins/send-email')
const createTestEmail = require('./cypress/plugins/create-account')
const parseEmail = require('./cypress/plugins/parse-email')

module.exports = defineConfig({
  chromeWebSecurity: false,
  e2e: {
    baseUrl: "https://dashboard.cypress.io",
    setupNodeEvents: async (on, config) => {

      on('task', {
        async createTestEmail() {
          const testAccount = await createTestEmail()
          return testAccount
        },
        async getLastEmail({ user, pass }) {
          const get_Email = await getLastEmail(user, pass)
          return get_Email
        },
        async sendEmail({ user, pass, emailObject }) {
          const send_Email = await sendEmail(user, pass, emailObject)
          return send_Email
        },
        async parseEmail({ message }) {
          const parse_Email = await parseEmail(message)
          return parse_Email
        }
      })
    },
  },
});

Write your tests...

spec.cy.js

/// <reference types="cypress" />
const { recurse } = require('cypress-recurse')

describe('Email confirmation', () => {
  let userEmail
  let userPass

  beforeEach(() => {
    recurse(
      () => cy.task("createTestEmail"),
      Cypress._.isObject, // keep retrying until the task returns an object
      {
        log: true,
        timeout: 20000, // retry up to 20 seconds
        delay: 5000, // wait 5 seconds between attempts
        error: "Could not create test email"
      }
    ).then((testAccount) => {
      userEmail = testAccount.user
      userPass = testAccount.pass
      cy.log(`Email account created - (for debugging purposes): ${userEmail}`)
      cy.log(`Email account password - (for debugging purposes): ${userPass}`)
    })
  })

  it('Fill in signup form and validate confirmation email is received', () => {
    cy.visit("/signup/email").pause()
    cy.get("#email").type(userEmail)
    cy.get("[type=password]").type(userPass)
    cy.get("button[type=submit]").click()

    cy.log("**redirects to /confirm**")
    cy.location("pathname").should("equal", "/verify")

    // retry fetching the email
    recurse(
      () => cy.task("getLastEmail", { user: userEmail, pass: userPass }), // Cypress commands to retry
      Cypress._.isObject, // keep retrying until the task returns an object
      {
        log: true,
        timeout: 30000, // retry up to 30 seconds
        delay: 5000, // wait 5 seconds between attempts
        error: "Messages Not Found"
      }
    ).then((message) => {
      cy.task("parseEmail", { message })
        .its("html")
        .then((html) => {
          cy.document().then(document => {
            document.body.innerHTML = html;
          });
        })
    })

    cy.log("**Email message content validation**")
    cy.get("h1").should("contain","Activate your account")
    cy.get("a.link-button").should("contain","Verify Email")
  })

  it("Send an email with attachment and validate", () => {
    let emailObject = {
      from: "'Fred Foo 👻' <foo@example.com>", // sender address
      to: "bar@example.com, baz@example.com", // list of receivers
      subject: "Hello ✔", // Subject line
      text: "Hello world?", // plain text body
      html: "<b>Hello world?</b>", // html body
      attachments: [
        {
          filename: "hello.json",
          content: JSON.stringify({
            name: "Hello World!"
          })
        }
      ]
    }

    cy.task("sendEmail", { user: userEmail, pass: userPass, emailObject: emailObject }).then((response) => {
      cy.log("The message id: " + response)
      Cypress.env("messageId", response)
    })

    recurse(
      () => cy.task("getLastEmail", { user: userEmail, pass: userPass }), // Cypress commands to retry
      Cypress._.isObject, // keep retrying until the task returns an object
      {
        log: true,
        timeout: 30000, // retry up to 30 seconds
        delay: 5000, // wait 5 seconds between attempts
        error: "Messages Not Found"
      }
    ).then((message) => {
      console.log("THE SOURCE")
      console.log(message)
      cy.task("parseEmail", { message: message })
        .its("attachments")
        .then((attachments) => {
          expect(attachments[0].filename).to.eq("hello.json")
        })
    })
  })
})

The code of the test is commented to describe exactly what each part does.

And that's it. Of course, the part related to "sending emails" from you app, you would need to do some work to create the ethereal account and have your app send the notification emails (the second scenario I mentioned at the beginning) to the ethereal account. Then fetch and validate

Hope this is clear enough and helpful... (And remember to use the Cypress Dashboard signup page only a couple of times if you want, just to understand how the code works. Let's try not to flood our Cypress folks' server with registration requests)

Cheers! and happy testing!