SQRT.app or how to cut test automation costs

Hello everyone, my name is Vitali, I am a Test automation engineer and Co-Founder of the sqrt.app. I've been in the IT sphere many years and managed to see a lot of "good" solutions, but today I would like to tell you why it should be useful for any engineer, manager, or tester to use this tool.

So, for starters, I would like to start with the question: "how and why did this solution exists?"

The answer is that the solution was born back when I came as a test automation engineer to one of the projects and there was test automation for Node.js. I had the opportunity to take part in the refactoring of the current solution and writing new scripts. After that project, there was another, and then another, and another, and a few more. Like any engineer, I had to solve similar problems, but already a little immersed in Java. And when studying automation for this language, I came to what we have.

Of course, you can tell me that there are BDD solutions where the user writes automation scripts in “business” language. However, the reality is a little disappointing - after all, if we remember how cucumber scripts are written, then we get something like this:

Feature: User Login
  As a registered user
  I want to log in to the website
  So that I can access my account
Scenario: Successful login
Given I am on the login page
When I enter valid credentials
And click the login button
Then I should be redirected to my account page

And the JS code will be something like this:

const { Given, When, Then } = require('cucumber');
const { expect } = require('chai');
const LoginPage = require('../pages/login.page');
const DashboardPage = require('../pages/dashboard.page');

Given('I am on the login page', async function () {
  await LoginPage.open();
});
When('I enter valid credentials', async function () {
  await LoginPage.login('username', 'password');
});
When('click the login button', async function () {
  await LoginPage.clickLoginButton();
});
Then('I should be redirected to my account page', async function () {
  expect(await DashboardPage.isOnDashboardPage()).to.be.true;
});

And here we are faced with the fact that a “business” language like gherkin cannot exist without a “code” description. Therefore, I dismissed this idea. Ideally, I would like the code to be as small as possible and at the same time be understandable to "non-coders".

We also recall that many testing frameworks are ported to several languages (like selenium, playwright) and they all have similar APIs. And here I would like to step aside and talk about the development of CI/CD pipelines in which tests are launched. Now many CI/CD vendors use the platform-independent markup language YAML (Yet another markup language), for example, this is how the github actions looks like.

name: Node.js CI
on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
jobs:
  build:
    runs-on: ubuntu-latest

strategy:
  matrix:
    node-version: [12.x, 14.x, 15.x]

steps:
  - uses: actions/checkout@v2
  - name: Use Node.js ${{ matrix.node-version }}
    uses: actions/setup-node@v2
    with:
      node-version: ${{ matrix.node-version }}
  - run: npm ci
  - run: npm run build --if-present
  - run: npm test
  - name: Upload coverage report to Codecov
    uses: codecov/codecov-action@v1
    with:
      token: ${{ secrets.CODECOV_TOKEN }}
      fail_ci_if_error: true

I hope that this piece of the code does not require explanation.

So, I came to the conclusion that steps in github actions are similar to steps in testing (one of the principles of testing is the presence of a final and predictable result)

Therefore, I present to your attention - sqrt.app

Why?

• In order not to write a boilerplate from project to project for automation (for example, often on projects, before interacting with an element, they write the function of scrolling to the interacting element).

• So that “non-coders” can understand automation without being tied to a specific PL.

• In order not to set up integrations for each project in a new way (I would like to do integrations out of the box, but so far this is in the plans)

• To make it easy to start automation on a project where there are no test automation engineers or not enough engineer qualification.

• To Visualize automation progress and it's status for managers and other non-engineers.

What does sqrt.app solve?

1. Programming language is an independent solution.
2. Mastering yaml and / or a visual graphical interface is easier than programming language with all its constructions (this may change during the project development)
3. Launch automation with just 1 console command and without much effort (only node.js is needed)

So, sqrt.app is a framework or a ready-made solution for your test automation.

How to install?

npm [e-mail]/sqrt

You can run both in “single threaded” mode and in a daemon (there is no docker yet, but we are working on it :) )

#! single threaded mode
square start
#! "daemon process"
sqrt start -d

By the way, running through a daemon process is very useful on the customer machine or ci/cd systems such as jenkins which runs 24/7.

You can kill the daemon process with the command:

sqrt down

The sqrt start command will create a .flows folder in the directory where your console is running. And if there is no such folder, then the start command will also create 2 default configurations (for running one script and for one group).

After executing the start command, you can open the UI and create a automation steps.

Before showing screenshots of the program itself, let's talk a little about entities.

There are 3 entities: configurations, groups, and scenarios. We will talk about each now:

• Configurations are a set of settings: "what exactly to run" and "with what parameters to run", for example headess: true.

YAML configuration example:

# filename: .flows/configurations/configuration-example.yml
id: configuration-example
scenarios:
  - "scenario-1" 
  - "scenario-2"
# ... rest scenarios
groups:
  - "group-1"
  - "group-2"
name: Custom configuration for many groups at once
settings:
  headless: true # launch browser in headless mode
  locale: "de-DE" # default is not set

• Groups are a set of scenarios united on some basis. Although the script may be without a group, however, this may be rare - then this script will not be displayed to the user on the UI.

The login page groups might look like this:

# filename: .flows/groups/group-login.yml
scenarios: # scenarios is required field
  - "scenario-login-success"
  - "scenario-login-failure"
name: Login group
id: group-login 
createdAt: 1670452907227 # auto calculated field
updatedAt: 1670452907227 # auto calculated field

• A test or scenario is the steps of tests: "what to open", "where to click", etc.

# filename: .flows/scenarios/example-test.yml
name: Example test
description: The internet app for input
createdAt: 1670693418092
updatedAt: 1679482575603
steps:
  - action: automation/web/session/new
  - action: automation/web/navigation/goto
    name: Open application
    params:
      url: https://the-internet.herokuapp.com
  - action: automation/web/element/click
    name: Navigate to the "Input challenge" page
    params:
      selector: a[href="/inputs"]
      waitForNavigation: true # useful if action trigger to new page
  - action: automation/web/element/find
    name: Get Input element
    params:
      selector: input[type="number"]
      saveAs: ${{inputElement}}
  - action: automation/web/element/typeText
    name: set Element value "123"
    params:
      value: 123
      selector: input[type=number]
    # 6 - запоминаем значение с инпута
  - action: automation/web/element/getInputValue
    name: Get input value
    params:
      saveAs: ${{inputValue}}
      selector: input[type=number]
    # 7 - проверяем что в инпуте сохранено значение 123
  - action: automation/assert/equal
    name: Expect that the input value is equal to "123"
    params:
      actual: ${{inputValue}}
      expected: "123"
  - action: automation/web/session/close

Let's see how this scenario will look on the UI. After executing the sqrt start command, you can open UI and see the script.

Once the script is written, it can be run either through the “debug” mode or through the configuration run. This is very convenient if you need to run a newly written test.

Once your test is ready, you can run it in "debug" mode and then see the progress on the current step and screenshots next to each step.

You can also run tests through configuration.

After selecting script files and/or groups, you can also set specific settings such as screen resolution, geolocation, and so on.

That's all I would like to show / tell in this review article.

If You are impressed - write us: [e-mail]

In anticipation of questions in the comments, I will immediately answer:

• The product is still raw and there are sooo many problems. Starting from installation and ending with the launch of the script.

• A lot of the world of the web is not implemented (for example, working with dialogs), but that's it for now.

• There is no docker yet, you need to measure it, although the documentation contains installation via docker, however, the image is not deployed.

• Yaml files are not supported with yaml extension (for now), only yml extension.

• Scenarios can be written both in "code" (in the yaml structure) and visually.

I look forward to reasoned criticism in the comments, and even better - feature requests and bug reports in our public issue tracker.

Documentation can be found here: https://sqrt.app/docs/get-started/installation/.

P.S. while the plans are huge and I want a lot and everything, but alas, there are only 24 hours in a day. We also cannot focus solely on bug fixes and new functionality due to the fact that we make the product “in the evenings”. We plan to enter SAAS so that we can feel the product without installing the package.

P.P.S. We are looking for projects who's found us interesting.