DEV Community

Cover image for Planning Your Next Tests with

Posted on

Planning Your Next Tests with

It is so easy to start writing tests with It seems that Cypress opened doors to testing for everyone with basic JavaScript knowledge. However, when the number of tests grows writing new tests becomes harder. And the issue is not just about writing code. Planning new test scenarios, refactoring, configuring pipelines, and getting test run reports. The more people involved in the product development the more visibility the testing process needs.

To keep end 2 end tests organized, following questions should be answered:

  • what tests do we have?
  • what parts of an application are covered with tests?
  • what new test scenarios should be implemented?
  • what is the current state for passed and failed tests?
  • if there are flaky tests on a project?

🤔 is just a test runner, but it can be paired with, a test management system for automated tests.
Let's see how can answer the questions risen above. We will use Cypress Realworld App, which is a real-world project (as you might have guessed) with a well-written Cypress tests.

Creating project in

🤔 What tests do we have?

But answering the very first question is not as simple as it seems.

All end 2 end tests are written as JavaScript code so to read them a person should be at least familiar with JavaScript, GitHub, etc. So this already clothes doors for business analysts, product managers, and manual QAs, for all these folks who do not actually code.

That's why we can import a Cypress realworld app into a newly created project on

Import project from source code in

Importing happens with a magical npx check-tests command which scans all the tests on a project and imports them into UI without executing them. This way can present those tests to non-tech folks.

To import only UI tests from realworld app project, we need to run check-tests command with --typescript flag (as the project is written in TypeScript) specifying that we use and the path to tests via --dir parameter:

TESTOMATIO=a9ae6798atoj npx check-tests@latest "**/*{.,_}{test,spec}.ts" --dir cypress/tests
Enter fullscreen mode Exit fullscreen mode

a9ae6798atoj is an API key of a project you are importing to, so in your case, it will be a different one.

If you use Import from Source button, you will get this command generated. Take note, that by default it imports tests from cypress/integration folder which doesn't exist in cypress-realworld project. So you need to change path and use cypress/tests instead.

Importing is fast, it takes just a few seconds to get this output.

Import output of project in

Then all imported tests can be reviewed.

Listed tests in

That's great that all tests are finally visible to team members. But is there a way they could be linked to specifications they are testing? Yes, and that answers the next question:

🤔 What parts of an application are covered with tests?

If the development management happens in Jira, Testomatio can link automated tests to Jira issue. You can link tests from a Cypress spec to an Issue:

Linking Test to Jira Issue in

Now all tests will be available within Jira, so can be taken into the development process. Revealing automated tests to Jira Issues helps managers and business analysts to understand more information on quality control. They don't need to leave Jira to see those automated tests!

Automated Tests in Jira Plugin

The only thing needed, is a Testomatio Jira plugin should be installed from a marketplace and a Jira Project to be connected to Testomatio project.

And we are almost ready to answer the next question:

🤔 What new test scenarios should be implemented?

A project manager or a business analyst knows what features are coming next so they are the right person to propose the next tests.

As they can't be asked to write a code, or open an issue for each test, they could create in test directly from Jira. After that, a developer can take the written scenario and implement it using

Writing Tests inside Jira plugin

As you see on the screenshot, step autocompletion can be used to help non-tech folks write scenarios efficiently.

The test with a description is treated as a manual test, but once it is implemented as Cypress test, and re-imported it will be marked as automated.

🤔 What is the current state for passed and failed tests can display and store Cypress reports. To send run reports to a @testomatio/reporter package should be installed and enabled in cypress/plugins/index.ts.

// inside cypress/plugins/index.ts
// require testomatio reporter package:
const testomatioReporter = require('@testomatio/reporter/lib/adapter/cypress-plugin');

export default (on, config) => {
  testomatioReporter(on, config);

  // ....
Enter fullscreen mode Exit fullscreen mode

This reporter can be added to any project, following instructions from

Setting up reporting in

Then tests should be executed via npx cypress run with few extra parameters:

TESTOMATIO=a9ae6798atoj npx cypress run
Enter fullscreen mode Exit fullscreen mode

a9ae6798atoj is an API key of a project to which reports will be sent.

A report will contain a list of passed and failed tests, with videos and screenshots. External S3 storage from one of the cloud providers (AWS, DigitalOcean, Google Cloud, etc) can be used to keep unlimited screenshots and videos.

Unlimited Test Artifacts for Tests in

Videos and screenshots can be enabled in Testomatio by adding S3 credentials into .env file of Cypress Realworld project:

Enter fullscreen mode Exit fullscreen mode

A complete historical data of all test executions are also available:

Run Reports in

If we have this data the final question can be answered:

🤔 Are there flaky tests on a project? has an analytics dashboard where all tests are analyzed to provide valuable insights. For instance, it is important to keep track of flaky tests, and slowest tests, and check for the most common defects across tests.

Flaky Tests Detection in

Final Thoughts

Developing end-2-end tests with is fun. But making them first-class citizens in project development is also important. Cypress End 2 end tests must be visible to the whole team, so even team members who are not directly involved in testing could track the progress, propose new changes to tests, and see the actual test coverage.

Testing is a crucial part of the development process. The more clear it is and the more team members are aware of it, the more quality software can be delivered. While helps build efficient and fast tests for the modern web, provides a feature-rich test management solution for all kinds of automated tests.

P.S. is absolutely free to use for personal projects. Try it out with Cypress realworld app, or any other Cypress project.

Top comments (3)

liviufromendtest profile image
Liviu Lupei

Cypress isn't really popular these days, mostly due to the limitations.

Such limitations include:

  1. Not being able to run tests on Safari (see What Happens When You Don't Test in Safari)
  2. Not being able to run tests in multiple browser tabs (see How to test in multiple tabs)
  3. No full support for iframes
davert profile image

I agree on the limitations. They are a lot of them!
However, Cypress fits well niche of e2e an component testing for modern SPA

For more complete End2End testing experience I would look into Playwright and CodeceptJS

liviufromendtest profile image
Liviu Lupei

Those libraries work great for folks looking to build a career out of writing tests.

Most developers I talked to just want to create the tests as fast as possible and be done with them, that's why they end up using platforms like Endtest or Mabl