Rest your eye with AyeSpy -- Automated visual testing

A lesser known but still useful tool to automate visual testing for your project -- Aye Spy can save precious time between development iterations.

Quick facts:

• Requires Selenium Grid to run
• Inspired by Wraith and Backstop -- both popular visual regression testing tools
• According to its creators, the USP of this tool is its performance improvement over other visual testing tools (runs 40 screenshot comparisons in a minute)

A sample test:

• Install Aye Spy

npm install -g aye-spy

• Install the Docker Selenium Grid image (Note that you will need Docker installed on your system already):

docker run -d -p 4444:4444  --name selenium-hub selenium/hub:3.141.59-titanium
docker run -d -P -p 5900:5900 --link selenium-hub:hub -v /dev/shm:/dev/shm selenium/node-chrome-debug:3.141.59-titanium

Run these, one after the other. These install images of Selenium Grid and the Chrome browser on the Docker machine, respectively.

• Run ayespy init to generate the initial config file. This includes basic test configuration and test scenarios.

This is a sample config file:

The gridUrl is a required field and specifies the address of the Docker Selenium grid image running on your computer. That is followed by the folders for baseline, test and difference screenshots respectively. report holds the generated report which is an html file.
scenarios holds the test scenarios.

• Run the test

ayespy snap --browser chrome --config ayespy-config.json --run "Home"

Home is the scenario label from the config file (ayespy-config.json here)
This creates a snapshot in a folder latest.

• Save this snapshot as a baseline for further tests:

ayespy update-baseline --browser chrome --config ayespy-config.json

This updates the baseline folder by copying the screenshot taken in the previous step to that folder.

• Compare the screenshots by running:

ayespy compare --browser chrome --config ayespy-config.json

For a passing test, there would be no difference between the baseline and test screenshots and hence, no screenshot will be saved in the generatedDiffs folder. Also, no report would be created in the report folder.

For a failing test however, both folders will be created and have new files illustrating the difference.

The html report is also written to the report file and looks like this:

Wins for Aye-spy:

• Straight-forward setup
• Simple documentation
• Support for various viewports
• AWS S3 support to save images to
• Screenshots could be taken for multiple branches; this speeds up finding issues.

What needs work:

• In order to run scripts before taking a screenshot, the run must have selenium-webdriver and By exposed.
• Safari is not supported
• Does not support switching contexts to iFrames

There are other visual testing tools aside from Aye-spy; try these:

1. BackstopJS and
2. good ol' Jest(jest-image-snapshot specifically).