DEV Community: sania-dsouza

Alexa custom skill workflow using Github Actions

sania-dsouza — Thu, 09 Dec 2021 05:24:05 +0000

My Workflow

Project link:

sania-dsouza / devto-gha-hack2021

DevTo - Github Actions : Hackathon 2021

This repo and project was created as an entry into the DevTo-Github Hackthon 2021, which focused on integrating Github Actions in a project.

Category chosen: Interesting IoT

Getting Started

Ensure you have node installed. Find out: node -v
Clone the repo : git clone
Install dependencies : npm install
This project uses Jovo to create the Alexa skill. Jovo provides a 'debugger' that helps to view and test interactions with Alexa. Run it using jovo run
Click Launch to start the app
The Nutri Planner app must open and allow you to interact with this skill

Note: The app currently just allows the user to choose from a given list of responses. This is extendable to accept any custom user input to mimic regular Alexa behavior.

Github Actions integration

The Github Actions workflow is the .github/workflows/node.js.yml file The workflow is designed to function…

View on GitHub

License: MIT

What is this about?

This project is the development of a custom Alexa skill using the Jovo platform that 'enables professional teams to build and run apps that work across smart speakers, the web, mobile, and more.'

The app is a nutrition planner that takes meal inputs from the user; the app currently accepts input from the user for breakfast, lunch and dinner and provides hard-coded replies for the same right now (see picture below as seen in the Jovo debugger)
While not implemented in this project, the goal is to save the user's input and count the number of calories consumed by the user in a day, for health monitoring.
Aside from creating the app and using Github Actions for the workflow, the skill was built and deployed to the Alexa Developer Console too for registration and further testing (a screenshot is attached later in this post).
Now for where Github Actions comes in!
The following Actions have been used throughout the workflow:
~ checkout@v2 : To checkout code
~ setup-node@v2 : To get node installed
~ upload-artifact@v2 : To upload important artifacts such as the whole skill code and test reports to the workflow run
~ codecov-action@v2 : To report test coverage code to Codecov -- the code coverage tool
~ appleboy/lambda-action@master : To deploy zipped code to AWS Lambda
A number of secrets have been used to run the workflow as expected such as the AWS_ACCESS_KEY_ID, AWS_REGION etc.
The list of jobs in the workflow are below:
~ build
~ static-code-analysis
~ unit-test
~ code-coverage
~ virtual-alexa-tests
~ deploy-skill
~ store-artifacts

Codecov:

Jovo Debugger:

Github Actions run:

Submission Category :

Interesting IoT

Yaml File:

name: Node.js CI

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:
    runs-on: ubuntu-latest
    name: Build
    strategy:
      matrix:
        node-version: [14.x]

    steps:
    - uses: actions/checkout@v2
    - name: Use Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v2
      with:
        node-version: ${{ matrix.node-version }}
        cache: 'npm'
    - run: npm install

  static-code-analysis: 
    runs-on: ubuntu-latest
    name: Static code analysis 
    needs: build
    steps: 
    - name: Checkout
      uses: actions/checkout@v2
    - run: |
        npm install
        npm run eslint

  unit-test: 
    runs-on: ubuntu-latest
    name: Unit test using Jest
    needs: static-code-analysis
    steps: 
    - name: Checkout
      uses: actions/checkout@v2
    - run: |
        npm install
        npm run test
    - name: Upload results
      uses: actions/upload-artifact@v2
      with:
        name: unit-test-report
        path: reports/test-report.html

  code-coverage: 
    runs-on: ubuntu-latest
    name: Code Coverage using Codecov
    needs: unit-test
    steps:
    - name: Checkout
      uses: actions/checkout@v2
    - run: |
        npm install
        npm run codecov
    - name: Codecov push results
      uses: codecov/codecov-action@v2
      with:
          token: ${{ secrets.CODECOV_TOKEN }} 

  virtual-alexa-tests: 
    runs-on: ubuntu-latest
    name: Test on Virtual Alexa
    needs: code-coverage
    steps: 
    - name: Checkout
      uses: actions/checkout@v2
    - run: |
        npm install
        npm run test-virtual
    - name: Upload results
      uses: actions/upload-artifact@v2
      with:
        name: virtual-test-report
        path: reports/test-report.html  

  deploy-skill: 
    name: Build and deploy lambda
    needs: virtual-alexa-tests
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Install dependencies and zip folder contents
        run: |
          npm install && zip -r bundle.zip .
      - name: deploy zip to aws lambda
        uses: appleboy/lambda-action@master
        with:
          aws_access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws_secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws_region: ${{ secrets.AWS_REGION }}
          function_name: 'devto-gha-hack2021-dev-handler'
          zip_file: bundle.zip

  store-artifacts:
    name: Store skill code
    if: always()
    runs-on: ubuntu-latest
    needs: deploy-skill
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Upload code
        uses: actions/upload-artifact@v2
        with:
          name: code
          path: ${{ github.workspace }}

Additional Resources / Info

A screenshot of the skill in action on the Alexa Developer Console:

Author:

.ltag__user__id__278541 .follow-action-button { background-color: #a33bc9 !important; color: #000000 !important; border-color: #a33bc9 !important; }

sania-dsouzaFollow

..finding my balance between passion and fulfillment..

Automate emails using Google API

sania-dsouza — Sat, 05 Jun 2021 01:42:07 +0000

The Gmail API provides a neat way to automate email tasks such as reading and sending among others. This piece illustrates how you could read emails and create and send them, all automatically.

Automated emails could be used as a part of a larger process of automating routine maintenance tasks for example. In my case, the intent is to download a CSV file which is sent in an email, process it and send the result as an email.

Steps:

Set-up your system to work with the Gmail API
Read an email and download CSV from it
Create email body and send email to concerned recipient

First, Set-up the Gmail API

You can read about all of it from here. There are language-specific guides to enable the Gmail API on your Google ID; I chose Node.

Open the terminal in your machine or use a code editor (I am using VSCode). Familiarity with node is assumed.

Install the Google API library and required libraries:

npm install googleapis@39 --save
npm install fs readline

Create an initial file with the code below. Call this file index.js

// index.js

const fs = require('fs');
const readline = require('readline');
const {google} = require('googleapis');
const SCOPES = [
    'https://www.googleapis.com/auth/gmail.readonly',
    'https://www.googleapis.com/auth/gmail.modify',
    'https://www.googleapis.com/auth/gmail.compose',
    'https://www.googleapis.com/auth/gmail.send'
  ];
const TOKEN_PATH = 'token.json';

// Load client secrets from a local file.
fs.readFile('credentials.json', (err, content) => {
  if (err) return console.log('Error loading client secret file:', err);
  // Authorize a client with credentials, then call the Gmail API.
  authorize(JSON.parse(content), getAuth);
});

/**
 * Create an OAuth2 client with the given credentials, and then execute the
 * given callback function.
 * @param {Object} credentials The authorization client credentials.
 * @param {function} callback The callback to call with the authorized client.
 */
function authorize(credentials, callback) {
  const {client_secret, client_id, redirect_uris} = credentials.installed;
  // console.log(redirect_uris);
  const oAuth2Client = new google.auth.OAuth2(
      client_id, client_secret, redirect_uris[0]);

  // Check if we have previously stored a token.
  fs.readFile(TOKEN_PATH, (err, token) => {
    if (err) return getNewToken(oAuth2Client, callback);
    oAuth2Client.setCredentials(JSON.parse(token));
    callback(oAuth2Client);
  });
}

/**
 * Get and store new token after prompting for user authorization, and then
 * execute the given callback with the authorized OAuth2 client.
 * @param {google.auth.OAuth2} oAuth2Client The OAuth2 client to get token for.
 * @param {getEventsCallback} callback The callback for the authorized client.
 */
function getNewToken(oAuth2Client, callback) {
  const authUrl = oAuth2Client.generateAuthUrl({
    access_type: 'offline',
    scope: SCOPES,
  });
  console.log('Authorize this app by visiting this url:', authUrl);
  const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout,
  });
  rl.question('Enter the code from that page here: ', (code) => {
    rl.close();
    oAuth2Client.getToken(code, (err, token) => {
      if (err) return console.error('Error retrieving access token', err);
      oAuth2Client.setCredentials(token);
      // Store the token to disk for later program executions
      fs.writeFile(TOKEN_PATH, JSON.stringify(token), (err) => {
        if (err) return console.error(err);
        console.log('Token stored to', TOKEN_PATH);
      });
      callback(oAuth2Client);
    });
  });
}

function getAuth(auth) {
    //var check = new Check(auth);

    console.log("Auth'ed")
  }

Let's see what's happening here.

'Require' the libraries
Declare the scopes: these are the activities we are going to use the Gmail API for. There is a list of them in the documentation linked before.
Read the credentials for the Gmail account that is going to be used to send and read the emails. The credentials.json file is derived from Google Cloud Platform.
The following lines use the credentials from step 3 and then create an OAUTH2 client and authorize the client to perform the desired activity.

To create the `credentials.json` file:

Login to GCP using your account
Create a project (Click the little downward arrow near the Google Cloud Platform logo on the left)
Once the project is created, select it and then click Credentials on the sidebar.
From the top menu, select Create Credentials and OAuth Client ID from the resultant drop-down. You will have to configure a few things depending on your situation:
An app name
the scopes that you saw from the index.js file will have to be selected here also. For our needs, be sure to manually add the scopes if you don't see them in the table.

That didn't create a set of credentials if you have been following with the process. It just creates an application under the project. To create a set of credentials, select Create Credentials again. Then select the application type; I selected the Web application as I intended to use the email reader utility as a web app.

Add information pertaining to this new set of credentials. Make sure all the fields are filled as they are required for the utility to work. The fields have helpful tool-tips to guide you.

Click Create. A pop-up with the Client ID and Client Secret shows up. You can close the pop-up and instead select the Download button on the row created on your Credentials page to download the credentials JSON file.

Rename the file to credentials.json and move it to your project folder.

My file looks like this:

With that out of the way, you can now test the Gmail API setup.
[UPDATE: You will have to rename "web" in the credentials file to "installed"]

Run the file

node index.js

Follow the instructions on the terminal. You will have to grant access to the email ID you want to use and then get the code from the resultant page (or address bar) and paste that on the terminal. (Side note: This took me a while to figure out but look out for a code in the aforementioned places and you can proceed).

Once authentication is complete, you should see something like this:

A file token.json is now created in your folder which will be used hereafter by your application to read emails and send them.

Second, Read emails

Install the necessary libraries

npm install js-base64 cheerio open dotenv https fs mailparser

Create another file readMail.js

// readEmail.js

const {google} = require('googleapis');
var base64 = require('js-base64').Base64;
const cheerio = require('cheerio');
var open = require('open');
const dotenv = require('dotenv');
const https = require('https');
const fs = require('fs');
var Mailparser = require('mailparser').MailParser;

dotenv.config();

class Check{

    //auth is the constructor parameter.
    constructor(auth){
        this.me = process.env.GMAIL_USER;
        this.gmail = google.gmail({version: 'v1', auth});
        this.auth = auth;
    }

    //Check for emails
    checkForEmails(){
        var query = "from:support@example.com is:unread";
        // console.log(this.me);
        this.gmail.users.messages.list({
            userId: this.me,
            q: query 
        }, (err, res) => {
            if(!err){
                //mail array stores the mails.
                var mails = res.data.messages;
                // console.log(mails);
                this.getMail(mails[0].id);
                // console.log(mails[0].id)
            }
            else{
                console.log(err);
            }
        });        
    }

    // read mail 
    getMail(msgId){
        //This api call will fetch the mailbody
        this.gmail.users.messages.get({
            userId: this.me,
            id: msgId
        }, (err, res) => {
            if(!err){
                // console.log(res.data.payload);
                var body = res.data.payload.body.data;
                var htmlBody = base64.decode(body.replace(/-/g, '+').replace(/_/g, '/'));
                // console.log(htmlBody);
                var mailparser = new Mailparser();

                mailparser.on("end", (err,res) => {
                    if(err) {
                        console.log(err);
                    }
                })

                mailparser.on('data', (dat) => {
                    if(dat.type === 'text'){
                        const $ = cheerio.load(dat.textAsHtml);
                        var links = [];
                        // Get all links in the HTML
                        $('a').each(function(i) {
                            links[i] = $(this).attr('href');
                        });

                        console.log("Email read!");
                        // You can further process the email and parse for necessary information
                    }
                })

                mailparser.write(htmlBody);
                mailparser.end();
            }
        });
    }

module.exports = Check;

What is happening here?

Require the libraries
Initialize a .env file which stores your Gmail Username and password. This is then used in the Check class constructor.
Unread emails from the address support@example.com are checked for. Replace this with the sender whose emails you want to read. Here, the first email (the latest) will be read.
The mail body of the first email is read. You can parse and process this mail body.

But how would you run this file? Make a few changes to the index.js file. The updated code is this:

// index.js

const fs = require('fs');
const readline = require('readline');
const {google} = require('googleapis');
const Check = require('./readEmail');
const SCOPES = [
    'https://www.googleapis.com/auth/gmail.readonly',
    'https://www.googleapis.com/auth/gmail.modify',
    'https://www.googleapis.com/auth/gmail.compose',
    'https://www.googleapis.com/auth/gmail.send'
  ];
const TOKEN_PATH = 'token.json';

// Load client secrets from a local file.
fs.readFile('credentials.json', (err, content) => {
  if (err) return console.log('Error loading client secret file:', err);
  // Authorize a client with credentials, then call the Gmail API.
  authorize(JSON.parse(content), getAuth);
});

/**
 * Create an OAuth2 client with the given credentials, and then execute the
 * given callback function.
 * @param {Object} credentials The authorization client credentials.
 * @param {function} callback The callback to call with the authorized client.
 */
function authorize(credentials, callback) {
  const {client_secret, client_id, redirect_uris} = credentials.installed;
  // console.log(redirect_uris);
  const oAuth2Client = new google.auth.OAuth2(
      client_id, client_secret, redirect_uris[0]);

  // Check if we have previously stored a token.
  fs.readFile(TOKEN_PATH, (err, token) => {
    if (err) return getNewToken(oAuth2Client, callback);
    oAuth2Client.setCredentials(JSON.parse(token));
    callback(oAuth2Client);
  });
}

/**
 * Get and store new token after prompting for user authorization, and then
 * execute the given callback with the authorized OAuth2 client.
 * @param {google.auth.OAuth2} oAuth2Client The OAuth2 client to get token for.
 * @param {getEventsCallback} callback The callback for the authorized client.
 */
function getNewToken(oAuth2Client, callback) {
  const authUrl = oAuth2Client.generateAuthUrl({
    access_type: 'offline',
    scope: SCOPES,
  });
  console.log('Authorize this app by visiting this url:', authUrl);
  const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout,
  });
  rl.question('Enter the code from that page here: ', (code) => {
    rl.close();
    oAuth2Client.getToken(code, (err, token) => {
      if (err) return console.error('Error retrieving access token', err);
      oAuth2Client.setCredentials(token);
      // Store the token to disk for later program executions
      fs.writeFile(TOKEN_PATH, JSON.stringify(token), (err) => {
        if (err) return console.error(err);
        console.log('Token stored to', TOKEN_PATH);
      });
      callback(oAuth2Client);
    });
  });
}

function getAuth(auth) {
    var check = new Check(auth);

    console.log("Auth'ed");
    check.checkForEmails();
  }

Run the file again

node index.js

Output on the console:

You can do a number of things with the resultant mail body like parse it, get a download link etc.

Kudos on getting here! Now for the last part: create an email and send it!

Third, Compose email and send

Install the library

npm install nodemailer

Create a file composeEmail.js and copy this code :

// composeEmail.js

const {google} = require('googleapis');
const mailComposer = require('nodemailer/lib/mail-composer');
var base64 = require('js-base64').Base64;
const dotenv = require('dotenv');
dotenv.config();

class CreateMail{

    constructor(auth, to, sub, body){
        this.me = process.env.GMAIL_USER;    
        this.auth = auth;
        this.to = to;
        this.sub = sub;
        this.body = body;
        this.gmail = google.gmail({version: 'v1', auth});
    }

    // Construct the mail
    makeBody(){

        let mail = new mailComposer({
            to: this.to,
            text: this.body,
            subject: this.sub,
            textEncoding: "base64"
        });

    //Compiles and encodes the mail.
    mail.compile().build((err, msg) => {
    if (err){
        return console.log('Error compiling email ' + error);
        } 

    const encodedMessage = Buffer.from(msg)
              .toString('base64')
              .replace(/\+/g, '-')
              .replace(/\//g, '_')
              .replace(/=+$/, '');


        this.sendMail(encodedMessage);
      });
       }

    //Send the message to specified receiver
    sendMail(encodedMessage){
        this.gmail.users.messages.send({
            userId: process.env.GMAIL_USER,
            resource: {
                raw: encodedMessage,
            }
         }, (err, result) => {
            if(err){
                return console.log('NODEMAILER - Returned an error: ' + err);
            }

            console.log("NODEMAILER - Sending email reply:", result.data);
        });
    }
}

module.exports = CreateMail;

What's happening here ?

'Require' the libraries
Construct the email body using base64
Send the email using nodemailer to the recipient selected

But again, how would you run this? Let's update the file readEmail.js to call composeEmail.js. The final code for readEmail.js is below:

const {google} = require('googleapis');
var base64 = require('js-base64').Base64;
const cheerio = require('cheerio');
var open = require('open');
const dotenv = require('dotenv');
const https = require('https');
const fs = require('fs');
var Mailparser = require('mailparser').MailParser;
const Email = require('./composeEmail');

dotenv.config();

class Check{

    //auth is the constructor parameter.
    constructor(auth){
        this.me = process.env.GMAIL_USER;
        this.gmail = google.gmail({version: 'v1', auth});
        this.auth = auth;
    }

    //Check for emails
    checkForEmails(){
        var query = "from:support@figma.com is:unread";
        // console.log(this.me);
        this.gmail.users.messages.list({
            userId: this.me,
            q: query 
        }, (err, res) => {
            if(!err){
                //mail array stores the mails.
                var mails = res.data.messages;
                // console.log(mails);
                this.getMail(mails[0].id);
                // console.log(mails[0].id)
            }
            else{
                console.log(err);
            }
        });        
    }

    // read mail 
    getMail(msgId){
        //This api call will fetch the mailbody
        this.gmail.users.messages.get({
            userId: this.me,
            id: msgId
        }, (err, res) => {
            if(!err){
                // console.log(res.data.payload);
                var body = res.data.payload.body.data;
                var htmlBody = base64.decode(body.replace(/-/g, '+').replace(/_/g, '/'));
                // console.log(htmlBody);
                var mailparser = new Mailparser();

                mailparser.on("end", (err,res) => {
                    if(err) {
                        console.log(err);
                    }
                })

                mailparser.on('data', (dat) => {
                    if(dat.type === 'text'){
                        const $ = cheerio.load(dat.textAsHtml);
                        var links = [];
                        var modLinks = [];
                        // Get all links in the HTML
                        $('a').each(function(i) {
                            links[i] = $(this).attr('href');
                        });

                        console.log("Email read!");
                    }
                })

                mailparser.write(htmlBody);
                mailparser.end();

                // Finally send the email 
                this.sendEmail("This is where the email's body goes.");
            }
        });
    }

    sendEmail(mail_body) {    
        var makeEmail = new Email(this.auth, <recipient_email_address>, 'Test subject', mail_body);

        // send the email
        makeEmail.makeBody();
    }
}

module.exports= Check;

Remember to replace the recipient's address in the sendEmail function above

Now run index.js

node index.js

This in turn runs readEmail.js which lastly runs composeEmail.js. Phew!

My console output looked like this:

And finally, the email!

Automating emails has many use cases and I hope this helped. Thanks and feedback is welcome!

Rest your eye with AyeSpy -- Automated visual testing

sania-dsouza — Sat, 27 Mar 2021 01:35:35 +0000

A lesser known but still useful tool to automate visual testing for your project -- Aye Spy can save precious time between development iterations.

Quick facts:

Requires Selenium Grid to run
Inspired by Wraith and Backstop -- both popular visual regression testing tools
According to its creators, the USP of this tool is its performance improvement over other visual testing tools (runs 40 screenshot comparisons in a minute)

A sample test:

Install Aye Spy

npm install -g aye-spy

Install the Docker Selenium Grid image (Note that you will need Docker installed on your system already):

docker run -d -p 4444:4444  --name selenium-hub selenium/hub:3.141.59-titanium
docker run -d -P -p 5900:5900 --link selenium-hub:hub -v /dev/shm:/dev/shm selenium/node-chrome-debug:3.141.59-titanium

Run these, one after the other. These install images of Selenium Grid and the Chrome browser on the Docker machine, respectively.

Run ayespy init to generate the initial config file. This includes basic test configuration and test scenarios.

This is a sample config file:

The gridUrl is a required field and specifies the address of the Docker Selenium grid image running on your computer. That is followed by the folders for baseline, test and difference screenshots respectively. report holds the generated report which is an html file.
scenarios holds the test scenarios.

Run the test

ayespy snap --browser chrome --config ayespy-config.json --run "Home"

Home is the scenario label from the config file (ayespy-config.json here)
This creates a snapshot in a folder latest.

Save this snapshot as a baseline for further tests:

ayespy update-baseline --browser chrome --config ayespy-config.json

This updates the baseline folder by copying the screenshot taken in the previous step to that folder.

Compare the screenshots by running:

ayespy compare --browser chrome --config ayespy-config.json

For a passing test, there would be no difference between the baseline and test screenshots and hence, no screenshot will be saved in the generatedDiffs folder. Also, no report would be created in the report folder.

For a failing test however, both folders will be created and have new files illustrating the difference.

The html report is also written to the report file and looks like this:

Wins for Aye-spy:

Straight-forward setup
Simple documentation
Support for various viewports
AWS S3 support to save images to
Screenshots could be taken for multiple branches; this speeds up finding issues.

What needs work:

In order to run scripts before taking a screenshot, the run must have selenium-webdriver and By exposed.
Safari is not supported
Does not support switching contexts to iFrames

There are other visual testing tools aside from Aye-spy; try these:

BackstopJS and
good ol' Jest(jest-image-snapshot specifically).

Test for visual regression with Jest-image-snapshot

sania-dsouza — Wed, 24 Mar 2021 15:54:12 +0000

Jest has a feature called snapshot testing where a serializable value for the React tree is generated and then compared with a reference snapshot to check for differences.

However, this article focuses instead on the more visual screenshot comparison that is provided by the jest-image-snapshot package.

Quick facts:

It's a Jest matcher that performs image comparisons using pixelmatch
jest-image-snapshot will not work with anything below Jest 20.x.x
Could add a Gaussian blur for noise
Once the snapshot is taken, it works exactly the same as Jest snapshots

Sample test:

This test assumes you have Jest installed and have basic Jest know-how. The project was built using Create React App. After this initial set-up, follow the steps below for visual testing goodness.

Install the package : ```

npm i --save-dev jest-image-snapshot


* Also install Puppeteer for user interaction

npm install puppeteer


* Test script:

import { toMatchImageSnapshot } from 'jest-image-snapshot';
const puppeteer = require('puppeteer');
expect.extend({ toMatchImageSnapshot });

it('CreateReactApp home', async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('http://localhost:3000');
const image = await page.screenshot();

expect(image).toMatchImageSnapshot();

})


* Run the test the first time:

npm run test

This test opens the page running on the localhost, takes a snapshot and saves it in the folder `_image_snapshots_`.  

![jest2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4uu7fv6rxrgbdoxx6maq.png)
![jest3](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l65kr2m563q7iukyz07j.png)

The screenshot:

![jest4](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0mp3kbevpmp49e9thjvf.png)

* Make a change in the source code and re-run the test. 
The test fails this time. 

![jest5](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s9k869xbf8iyf2jlx7ma.png)

Another sub-folder is created by the name `_diff_output_`.

![jest6](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nbh43wz1vc3rziemnf0g.png)

The difference between the snapshot and the reference snapshot is shown marked: 

![jest7](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uyx3jx7pobpt6v9tc0j4.png)

This is a simple test. You could simulate more complicated user flows and take screenshots of sections of pages rather than full pages also. 

#### The good stuff:

* Easy setup once Jest is installed 
* Follows the Jest scaffolding of tests and can easily be integrated with existing functional tests 
* Multiple configuration options from the API
* Could set image difference sensitivity percentage 

#### Other things:

* No support for Typescript
* Outdated reference snapshots have to be removed manually and do not get cleared by using the `-u` flag of Jest. There is an environment variable that can be set up to remove the outdated snapshots but this utility must be used with caution. 

There are other paths for visual testing greatness:
1. [BackstopJS](https://dev.to/saniadsouza/visual-regression-testing-with-backstopjs-5509)
2. [Aye-spy](https://dev.to/saniadsouza/rest-your-eye-with-ayespy-visual-testing-2a36)

Prevent visual regression bugs with BackstopJS

sania-dsouza — Mon, 22 Mar 2021 23:12:04 +0000

Testing could be a lot of work if you are responsible for the visual (or even functional) feels of a website. And picking out visual differences after seemingly-unrelated code changes could be a pain. BackstopJS is one of the tools that could help automate this bit.

Following is a short tutorial on how to set up Backstop for your node project.

Quick facts:

BackstopJS automates visual regression testing of a responsive web UI by comparing DOM screenshots over time.
It includes an in-browser reporting feature, which allows you to check layout settings for print and screen, test approving, inspection, etc
Docker rendering for cross-platform tests
Simulating user interactions using Puppeteer

A quick test:

Install BackstopJS :

npm install -g backstopjs

Generate a Backstop config file:

backstop init

The file generated is backstop.json. This has some default config settings that Backstop looks for when it runs.

Some of the important config properties:

id : used for screenshot naming
viewports : array of viewport sizes ; at least one must be specified
scenarios: specifies different user flows for example. Section of the config file is shown below.

This test config file specifies a single scenario. All it does is navigate to the URL against url.

Run the test

backstop test

This first test fails since it didn’t find a reference image to compare the test screenshot with.

The report generated on browser (since the ‘report’ property was set to ‘browser’ in the config file) looks like this:

You might've noticed that when you ran backstop init, a set of folders was created. One of these was bitmaps_test. This folder holds the test screenshots.

To make a test file the reference for future tests, run:

backstop approve

This 'approves' the previous test screenshot as a standard or reference to compare future test screenshots with. It copies the screenshot from the bitmaps_test folder to the bitmaps_reference folder.

Run the test again:

backstop test

This time, the test passes.

The corresponding browser report:

Another test(with user interaction)

Puppeteer is used to simulate user scenarios.
Add another scenario to the scenarios array in the config file:

...
"scenarios": [
    {
      "label": "BackstopJS Homepage",
      "cookiePath": "backstop_data/engine_scripts/cookies.json",
      "url": "https://garris.github.io/BackstopJS/"
    },
     {
       "label": "BackStopJS Interaction scenario",
       "url": "https://garris.github.io/BackstopJS/",
       "clickSelector": ".cta"
     }
  ],
...

Here, the URL as specified by url is opened and when the element specified by the selector clickSelector is available, it is clicked; and then Backstop takes a screenshot.

The first time you run backstop test, the test will fail as it doesn't find the reference image. Run backstop approve and backstop test like before to get your test to pass.

In my test, this second test fails as well because of a difference between the reference and the test screenshot. This is indicated by the hot pink text on the difference screenshot.

At this point, I could either fix what changed in the code or update my reference using backstop approve.

Backstop also has a scrubber utility that displays the difference between the reference and test screenshots by moving a mapper cursor, which is pretty intuitive.

You could automate more complex user flows, add more viewports, tune performance and other good stuff using Backstop; their documentation is great and they have a healthy user base also.

Backstop niceties:

Easy to set-up
Good support for various viewports and Docker integration in case of cross-platform issues
Intuitive reporting and inspection
Easy Puppeteer scripts to simulate user interaction
Variable image difference sensitivity

Not a con, but a housekeeping task:

Outdated screenshots will have to be cleared manually or pushed to a .gitignore file to ensure they don't make it into the remote repo.

PS: If you are anything (read curious) like me, that cutie on the Backstop logo is a ring-tailed lemur ;).

Try these visual testing tools too:

Jest(jest-image-snapshot specifically)
Aye-spy

Artillery -- Quick check your site's performance

sania-dsouza — Fri, 19 Mar 2021 01:14:00 +0000

Jmeter is a great performance testing tool but has a steep learning curve and could take long to set up. Sometimes, there is just not enough time for that. Come in Artillery !

Quick facts:

Scripts written in YAML: which greatly reduces the code knowledge you need to have to set up tests.
Designed for testing backend systems, such as API services, e-commerce backends, chat systems, game backends, databases, message brokers and queues, and anything else that can be communicated with over a network.
Can't test frontends
Two options: Artillery Core (free) and Artillery Pro (paid)
Functional and load testing could be performed in one package
Artillery Pro is used to run the performance tests on the Cloud i.e., AWS.

Simple installation using NPM:

npm install -g artillery@1.6

A sample test

This test (let's say it's called test-artillery.yml) :

runs against the URL: https://blazedemo.com/,
follows a phased ramp-up/ramp-down set of steps:

       Warm-up : creates 1 virtual user each second for 5 seconds
       Ramp-up : creates 1 virtual user per second ramping up to 5 virtual users per second for 1 minute (60 seconds)
       Ramp-down : ramps down the number of concurrent virtual users to 0 over 15 seconds

asserts the aggregate p95 (95th percentile) latency is 200ms or less, and that the maximum error rate was less than 1%.
includes a scenario which is used to test a virtual user flow

config :
  target: "https://blazedemo.com/"
  plugins: 
    expect : {}
  phases:
    - duration: 5
      arrivalRate : 1
      name : Warm-up
    - duration: 60 
      arrivalRate: 1
      rampTo: 5
      name: Ramp up load
    - duration: 15
      arrivalRate: 1
      rampTo: 0
      name: Kill
  ensure:
    p95: 200
    maxErrorRate: 1
scenarios:
  - name: 'test scenario'
    flow:
      - get: 
          url: "/vacation.html"
          expect: 
            - statusCode: 200
      - think: 2

Run the test

artillery run test-artillery.yml

A section of the full console output

Console output is not the most easily understood report of results so Artillery has a built-in html reporting feature which is easier on the eyes.

First, create the report:

artillery run --output report.json test-artillery.yml

Sample JSON generated

Next, display that JSON as HTML:

artillery report --output report.html report.json

Sample HTML report

There is a ton of other stuff that Artillery can do including more complex user scenarios, running large-scale load tests, adding synthetic traffic in production to maintain a margin of safety against traffic spikes etc.
This post only skimmed over Artillery Core; Artillery Pro which is a paid service, provides a seamless upgrade path from tests running on a developer’s machine, to scaling up & running the same test scripts from your organization’s AWS account.

What's to like about Artillery

Scripts written in YAML; easy to learn
Easy to setup and good documentation
Good starting point for performance evaluation

What could improve

Could do with more options for reporting; it currently supports Datadog (via agent or HTTP API), StatsD and InfluxDB with Telegraf + StatsD plugin while others like Prometheus are in progress.

Happy testing!

How to use environment variables at build using Github Actions

sania-dsouza — Mon, 25 Jan 2021 18:10:08 +0000

Hi!

I recently ran into an issue when I needed to apply a continuous integration pipeline to my Gatsby portfolio site on Github. Github Actions seemed like a natural choice due to its ease of set-up. I used their readymade Node workflow action.

Setting it up this way creates a .yml file that is then directly added to your root directory on your repository on the .github/workflows path.

This is what it initially looks like.

name: Node.js CI

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [10.x, 12.x, 14.x, 15.x]
        # See supported Node.js release schedule at https://nodejs.org/en/about/releases/

    steps:
    - uses: actions/checkout@v2
    - name: Use Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v1
      with:
        node-version: ${{ matrix.node-version }}
    - run: npm ci
    - run: npm run build --if-present
    - run: npm test

My problem:

My site uses the Github API to pull my repos from my Github account. For this, I needed a personal access token. See how to create one here. This value was placed in a .env file on my root folder which was ignored and not sent to Github (via a .gitignore file). This was for obvious security reasons.

As a result, my workflow would fail at build when I pushed code, since the Github API couldn't find the key it needed to build the page.

Solution:

The way to get around this was to make the environment variable in my .env to be available to the build without risking security.

For this, I used an action Create .env file which created an .env file at build and had a scope that would expire at build completion.

# This workflow will do a clean install of node dependencies, build the source code and run tests across different versions of node
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actions

name: Node.js CI

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [13.x]
        # See supported Node.js release schedule at https://nodejs.org/en/about/releases/

    steps:
    - uses: actions/checkout@v2
    - name: Use Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v1
      with:
        node-version: ${{ matrix.node-version }}

    - name: Create .env file
      uses: SpicyPizza/create-envfile@v1
      with:
        envkey_GITHUB_LOGIN: "sania-dsouza"
        envkey_GITHUB_PERSONAL_ACCESS_TOKEN: ${{ secrets.SECRET_KEY }}

    - run: npm ci
    - run: npm run build
    - run: npm test

GITHUB_PERSONAL_ACCESS_TOKEN is the name of the key that the Github API is expecting to use in my code.

I used the Secrets option on my repo to save the said token and referred to that secret (SECRET_KEY) in the .yml as above. Remember to not use GITHUB_ as a prefix while naming your secrets as those are reserved.
Read this for how to set secrets in Github.

The result -- a passing build !

Another solution would be to manually create a .env file at build (touch .env, push those environment variables to the file, cat .env).

Hope this alleviates some of the pain in setting up a secure Github Actions pipeline for you. :)

Also, feedback is most welcome.

DEV Community: sania-dsouza

Alexa custom skill workflow using Github Actions

My Workflow

sania-dsouza / devto-gha-hack2021

DevTo - Github Actions : Hackathon 2021

Getting Started

Github Actions integration

What is this about?

Submission Category :

Interesting IoT

Yaml File:

Additional Resources / Info

sania-dsouzaFollow

Automate emails using Google API

Steps:

First, Set-up the Gmail API

Let's see what's happening here.

To create the credentials.json file:

Second, Read emails

Third, Compose email and send

Rest your eye with AyeSpy -- Automated visual testing

Quick facts:

A sample test:

Wins for Aye-spy:

What needs work:

Test for visual regression with Jest-image-snapshot

Quick facts:

Sample test:

Prevent visual regression bugs with BackstopJS

Quick facts:

A quick test:

Another test(with user interaction)

Backstop niceties:

Not a con, but a housekeeping task:

Artillery -- Quick check your site's performance

Quick facts:

Simple installation using NPM:

A sample test

Run the test

A section of the full console output

Sample JSON generated

Sample HTML report

What's to like about Artillery

What could improve

How to use environment variables at build using Github Actions

To create the `credentials.json` file: