How to Perform E2E Testing Using Cypress CLI and Test Runner

End-to-End (E2E) testing is a software testing methodology that focuses on testing an application’s workflow from start to finish, simulating real user interactions, and verifying that all components of the system work together correctly.

To illustrate E2E testing, let’s consider a live scenario involving an eCommerce website. The goal is to test the entire purchasing process, from when a user adds items to the shopping cart until the order is successfully placed. We usually follow the below approach.

1. Test Setup: Ensure the testing environment is prepared with the latest version of the application and necessary test data.

2. Test Case/Automation script Preparation: Define test cases/scripts to cover the complete order placement process, including adding items to the cart, user authentication, shipping selection, payment processing, and order confirmation.

3. Test Execution: Begin the test by simulating user interactions, such as adding items to the cart, filling in user details, selecting shipping options, entering payment information, and confirming the order.

4. Test Completion: Review the test results, identify critical issues, and summarize the overall findings, including bugs, errors, or unexpected behavior.

   Selenium WebdriverIO is Javascript based test automation framework built over nodeJs. Learn with this guide how to use webdriverIO to perform web automation testing.

By performing E2E testing in a live scenario like this, you can gain confidence in the reliability and functionality of the eCommerce website’s purchasing process.

Running E2E tests using Cypress CLI helps you configure test runs, generate reports, and integrate with other tools and services to streamline your testing process. It provides a command-line interface to manage and execute E2E tests efficiently. The Cypress CLI provides a comprehensive set of commands and features that empower developers and testers to perform robust and efficient E2E testing.

The other way to run E2E tests is using Cypress Test Runner, which provides an interactive graphical interface that allows developers and testers to run their tests, monitor their progress, and analyze the results in real-time. When using the Cypress Test Runner, you can select specific test files or test suites to execute and then observe the test execution step by step.

You will learn more about different ways of running tests in Cypress, i.e., using Cypress CLI and Test Runner in the later sections of this Cypress tutorial.

What is E2E Testing?

End-to-End (E2E) testing is a software testing strategy that focuses on verifying the complete flow of an application from start to finish. It aims to ensure that all system components, including user interfaces, APIs, databases, and integrations, work together properly and meet the desired functionality.

It is often performed after unit testing and integration testing, which test individual components and their interactions, respectively. By conducting E2E testing, the entire system is evaluated as a whole, simulating real-world user scenarios and ensuring that the application behaves as expected in a production-like environment.

The primary goal of E2E testing is to validate the application’s functionality, reliability, and performance from end to end. It helps uncover any defects or issues arising from the interactions between different components. By thoroughly testing the application’s complete flow, E2E testing provides confidence that the system works as intended and meets the requirements of the end-users or business.

Let’s take an example of a simple E2E test scenario where you login into LambdaTest’s eCommerce Playground with valid credentials. After login subscribe to the newsletter and verify whether the subscription was successful.

Which are the most wanted automation testing tools in 2023 that have climbed the top of the ladder so far? Let’s take a look.

Why Cypress for End-to-End Testing?

E2E testing is a type of software testing that verifies the entire application from start to finish. This means that E2E tests simulate real user interactions with the application, ensuring that all of the application’s features work together as expected. Cypress is the best fit for E2E testing for several reasons:

• Ease of Use

Cypress provides an intuitive and easy-to-use API, which makes it user-friendly and allows developers/testers to write tests quickly. Its syntax is straightforward, and the documentation is extensive and well-maintained.

• Automatic waiting and real-time reloading

Cypress automatically waits for DOM elements to become available and actions to complete, eliminating the need for manual waits or timeouts. It also provides real-time reloading, which means developers/testers can see the impact of their code changes immediately during test execution.

• Automatic screenshots and video recording

Cypress automatically captures screenshots and records videos during test runs. This is helpful for visual debugging and for documenting test failures, making it easier to understand and reproduce issues.

• Built-in Test Runner

Cypress comes with a built-in Test Runner, which means there is no need to set up or configure additional tools or frameworks. The Test Runner provides a clear interface to view test results, manage test files, and execute tests with a simple command.

• Active and supportive community

Cypress has a vibrant and active community of developers who contribute to its development and provide support. The official documentation is comprehensive and regularly updated, and there are numerous community-written tutorials, blog posts, and plugins available

• Full control over the application

To understand Cypress E2E testing, let’s take a closer look at the test automation pyramid. We routinely do each level of testing listed in this pyramid while performing Cypress testing.

Deep dive to learn what is automation testing, its uasage, types and also gain insights on how to get started with automated testing.

Subscribe to the LambdaTest YouTube Channel to get the latest updates on tutorials around Selenium testing, Cypress e2e testing, Appium, and more.

Testing Pyramid Layers in Cypress

Cypress team introduced component testing with Cypress 10 that allows developers to test individual components quickly, regardless of their complexity. Component tests differ from E2E tests in that instead of visiting a URL to pull up an entire app; a component can be “mounted” and tested on its own.

In the context of Cypress, here’s how the testing pyramid can be applied:

Unit Tests

These tests focus on testing individual components or functions in isolation. In Cypress, you can use frameworks like Mocha or Jest to write unit tests for your JavaScript functions.

These tests are typically executed quickly and frequently, ensuring that each component of your application works as expected.

Component Tests

These tests verify the behavior of individual components or UI elements in your application. In Cypress, you can write component tests that interact with specific UI elements and verify their functionality.

These tests ensure that each component functions correctly within the application’s context.

Integration Tests

Integration tests focus on testing the interactions between different components or modules in your application. In Cypress, you can write integration tests that simulate user interactions and verify the integration of various components.

These tests ensure that the different parts of your application work together seamlessly.

End-to-End (E2E) Tests

These tests span the entirety of the application’s process, validating its behavior holistically. Within Cypress, E2E tests can be authored to replicate user actions, engage with the UI, and authenticate anticipated results.

Although these tests often demand more time for execution, they deliver a sense of assurance in the comprehensive functionality of your application.

To know more about component testing, let’s see the difference between end-to-end testing and component testing.

This blog discusses the most exciting features of 13 best automation testing frameworks in 2021.

Here is an example of Cypress E2E test scenario:

Test Scenario

1. Visit the site

https://ecommerce-playground.lambdatest.io/index.php?route=account/login

1. Search with valid data and verify the search data should be displayed.

2. Search with invalid data and verify the message.

Implementation

In the following scenario, we will navigate to the website and perform searches using both valid and invalid data.

describe('Search  with valid and Invalid data' , () => {
 beforeEach(() => {
 cy.visit('https://ecommerce-playground.lambdatest.io/index.php?route=account/login')
 })
 it('Searches for the valid text  "Apple" and  verify the results', () => {
  // Enter the search data and submit the form
   cy.get('[name="search"]').eq(0).type('Apple')
   cy.get('.type-text').click()
   // Verify the search results
   cy.url().should('include', 'search=Apple')
   cy.contains('Search - Apple').should('be.visible')
   cy.get('.product-layout').should('have.length.gt', 0)
 })
 it('Displays message with no search results for invalid search term', () => {
   // Enter search term and verify returns no results and submit form
   cy.get('[name="search"]').eq(0).type('xyz')
   cy.get('.type-text').click()
   // Verify message for no search results
   cy.contains('There is no product that matches the search criteria.').should('be.visible')
 })
})

Let’s execute the test cases in Searchwith_validAndinvalid_data.cy.js.

Execution

Discover the 17 key benefits of automation testing, which will help you decide whether this is the right call for your SDLC.

What is Cypress CLI (Command Line Interface)?

CLI stands for Command-Line Interface. It is a text-based interface used to interact with a computer operating system or software by typing commands instead of using a graphical user interface (GUI).

Typically, CLI commands consist of a command followed by optional parameters or arguments. The user enters the command, and the system responds with the requested action or provides feedback on the command’s execution. CLI is popular among developers and system administrators due to its efficiency, flexibility, and ability to automate tasks through scripts or batch files.

Some well-known examples of CLI interfaces include the Windows Command Prompt (cmd.exe), PowerShell (for Windows), the Unix/Linux shell (such as Bash), and the macOS Terminal (based on the Unix shell).

The Cypress CLI provides a set of commands that enable developers and testers to interact with the WebElements and control the Cypress testing framework from the command line. It allows you to initialize new Cypress projects, run tests, open the Cypress Test Runner, and perform various other testing-related tasks.

With the Cypress CLI, you can perform various tasks, including:

• Opening the Cypress Test Runner: The CLI allows you to open the Cypress Test Runner, which provides a graphical user interface for running and debugging your tests. You can use this interface to select and execute individual test files or suites, view test results, and debug failing tests.

• Running tests in headless mode: Using Cypress CLI, you can run Cypress test automation in headless mode, which means the tests run without displaying the graphical user interface of the Test Runner. This is useful for running tests in a Continuous Integration (CI) environment or as part of an automated test suite.

• Running tests in different browsers: The Cypress CLI lets you specify the browser in which you want to run your Cypress tests. You can choose from popular browsers like Chrome, Firefox, or Electron or even specify custom browser configurations.

• **Running tests in parallel: **If you have a large suite of tests, the Cypress CLI enables you to run Cypress parallel testing across multiple instances of Cypress. This can significantly reduce the overall test execution time.

• Generating test reports: You can generate HTML or JSON reports for your test runs using the Cypress CLI. These reports provide detailed information about the test results, including passed and failed tests, execution times, and error messages.

• Managing configuration options: The Cypress CLI allows you to configure various options for Cypress, such as specifying the test file patterns to include or exclude, setting environment variables, configuring plugins, and more.

  This article on the Best UI Automation Testing Tools cover what is UI automated testing, challenges while performing UI testing, and top tools that can help you perform UI testing.

Here are some common commands we can run using Cypress CLI:

What is a Cypress Test Runner?

Cypress also comes with a built-in Test Runner GUI that provides an interactive interface for running and debugging tests. To launch the Test Runner, you can use the Cypress open command. This opens a Test Runner window where you can select and run individual tests, view test results, and debug your code.

The Cypress Test Runner operates directly in the browser and runs tests within it, enabling real-time visibility into the application under test. It offers a built-in time-traveling feature that allows developers to inspect and debug the application’s state at any point during the test execution. This makes it easier to troubleshoot and identify the causes of test failures.

Some key features of Cypress Test Runner include:

• Interactive Test Runner: Cypress Test Runner has an interactive user interface that allows you to view the test execution in real time. You can see the commands being executed, their results, and any errors or failures. This live feedback makes it easy to identify and diagnose issues during test execution.

• Time Travel: Cypress allows you to “time travel” through your application and see exactly what happens at each step of your test. You can pause the test execution, step forward or backward, and inspect the application’s state at any time.

• Debugging with DevTools: Cypress integrates with the browser’s Developer Tools, allowing you to debug your application and tests directly from the Cypress Test Runner. You can set breakpoints, inspect variables, step through code, and interact with the application while the test is running.

• Real-Time Reloads: Cypress automatically detects changes made to the test code and reloads them in the Test Runner interface. This live reload functionality provides a seamless development experience, allowing developers to see the immediate effects of their code changes without manually reloading the page.

• Network Stubbing and Spying: Cypress allows developers to stub or intercept network requests made by the application. This enables you to simulate different network scenarios, control the responses from APIs, and test edge cases.

Below is the screenshot of Cypress Runner, which opens when the user runs the command ‘yarn cypress open’.

Test native, hybrid, and web apps on any mobile OS with our free online emulator Android. Sign up to optimize app performance.

Running Test Cases in Cypress

When it comes to running Cypress test cases, there are different approaches you can take depending on your specific requirements and testing environment.

Here we will be discussing how you can run Cypress test cases using:

• Cypress CLI

• Cypress Test Runner

The Cypress CLI provides a straightforward way to run Cypress automation from the command line. You can use the cypress run command to run your tests in headless mode or cypress open to run them in interactive mode. The Cypress CLI also offers various options and flags to customize the test run, such as specifying specific test files, setting environment variables, and generating test reports.

Now let’s take some examples to execute the test cases in Cypress CLI.

Test Scenario

1. Visit the Site https://ecommerce-playground.lambdatest.io/index.php?route=account/login

2. Login into the site valid credential.

3. Do subscribe to the newsletter.

4. Verify newsletter is successfully subscribed.

Implementation

Let’s create a new folder under the e2e folder named “demoCypressCLIRunner” to perform Cypress UI testing.

Create the first test spec to check the login feature for the site with the name subscriptionScenario..cy.js under the folder ‘demoCypressCLIRunner’.

For demo purposes, we are using the LambdaTest eCommerce Playground site.

Use Case

Login into the application and update the newsletter subscription.

describe('Login and Subscribe', () => {
 it('Logs in and subscribes to newsletter', () => {
   // Visit the web site
   cy.visit('https://ecommerce-playground.lambdatest.io/index.php?route=account/login')


// Click on login button
 cy.get('[value="Login"]').click()
// Enter the valid email and password
cy.get('#input-email').type('[e-mail]')
cy.get('#input-password').type('Lambda123')


// Click on login button
cy.get('[value="Login"]').click()


// Verify user logged in successfully
cy.url().should('include', 'index.php?route=account/account')
cy.contains('My Account').should('be.visible')


// Subscribe the newsletter
 cy.contains('Newsletter').click()
 cy.get('#input-newsletter-yes').click({force:true})
 cy.get('[value="Continue"]').click()


 // Verify the subscription success message
   cy.get('.alert-success').should('contain', 'Success: Your newsletter subscription has been successfully updated!')
 })
})

Test native, hybrid, and web apps on any mobile OS with our free online android emulator. Sign up to optimize app performance.

How to Run Test Cases in CLI?

There are different ways to run the Cypress test cases in CLI.

Command 1:

Run all Cypress tests using Cypress CLI in the Electron default headless browser.

Output

All the tests are in headless mode in the default browser Electron. The Cypress command ran successfully, and our tests got executed.

Now, let’s understand some important aspects of this execution marked by the numbered tickets:

1. Command used to run all the test files present under the “e2e” folder.

2. Test cases started with Cypress, Browser, and Node versions with the number of test .spec files.

3. Test cases are executed one by one in the 3rd sticker. It shows that the first .spec file is passed.

4. Recording of all the test cases is saved under the ‘videos’ folder, which greatly helps during debugging.

5. Final test cases execution result report with parameters like Tests, Passing Test, Failing Test, Pending, and Skipped Tests.

A comprehensive end-to-end Testing tutorial that covers what E2E Testing is, its importance, benefits, and how to perform it with real-time examples.

Command 2:

Run the particular test in headless mode in the default browser Electron.

To run specific spec files from the list of test cases under the e2e folder, we can mention the path using “–spec” along with the “cypress run” command.

Output

In the output, we can see the test case execution report of a particular .spec.

Command 3:

Run all the tests in Cypress CLI to run tests in Chrome (Headless Mode).

Output

In the output, we can see the test case execution report of all test cases in the Chrome browser.

Command 4:

Run a particular test case in Cypress CLI to run tests in (Headless Mode).

Output

In the output, we can see the test case execution report of a particular .spec in the Chrome browser.

A comprehensive end-to-end Testing tutorial that covers what e2e Testing is, its importance, benefits, and how to perform it with real-time examples.
Let’s take one more scenario.

Test Scenario

1. Visit the site

https://ecommerce-playground.lambdatest.io/index.php?route=account/login

1. Search with valid data and verify the search data should be displayed.

2. Search with invalid data and verify the message.

Implementation

Create the second test case with the name “Searchwith_validAndinvalid_data.cy.js” under the LambdaTest folder.

In the below test case, we are covering the search with valid and invalid data and verifying the search result.

Use Case

Open the site and search with valid and invalid data.

describe('Search  with valid and Invalid data' , () => {
 beforeEach(() => {
 cy.visit('https://ecommerce-playground.lambdatest.io/index.php?route=account/login')
 })
 it('Searches for the valid text  "Apple" and  verify the results', () => {
  // Enter the search data and submit the form
   cy.get('[name="search"]').eq(0).type('Apple')
   cy.get('.type-text').click()
   // Verify the search results
   cy.url().should('include', 'search=Apple')
   cy.contains('Search - Apple').should('be.visible')
   cy.get('.product-layout').should('have.length.gt', 0)
 })
 it('Displays message with no search results for invalid search term', () => {
   // Enter search term and verify returns no results and submit form
   cy.get('[name="search"]').eq(0).type('xyz')
   cy.get('.type-text').click()
   // Verify message for no search results
   cy.contains('There is no product that matches the search criteria.').should('be.visible')
 })
})

Command 1:

Run all the tests in Cypress CLI in default browser Electron (Headless Mode).

yarn cypress run

Output

Command 2:

Run a particular test in Cypress CLI to run tests in Chrome (Headless Mode).

yarn cypress run --spec cypress/e2e/demoCypressCLIRunner/Searchwith_validAndinvalid_data.cy.js
Chrome

Output

Command 3:

Run all test cases in Cypress CLI to run tests in Chrome (Headless Mode).

yarn cypress run — headless — browser chrome

Output

Command 4:

Run particular test in headless mode in Chrome browser.

yarn cypress run - spec cypress/e2e/demoCypressCLIRunner/Searchwith_validAndinvalid_data.cy.js
Chrome

Output

In this Appium tutorial, learn about Appium and its benefits for mobile automation testing. Take a look at how Appium works and see how to perform Appium automation testing of your mobile applications.

How to Run Test Cases in Cypress Test Runner?

Cypress Test Runner provides a user interface (UI) that allows you to interact with and run your Cypress tests. After executing the appropriate command to start the Test Runner, the UI will be launched. From there, you have several options for running your tests.

By default, Cypress will launch the Test Runner UI with its default browser, which is typically Electron. However, you can switch to a different browser by selecting it from the available options in the browser drop-down menu.

To run a single test, just enter the particular .spec name and execute the test case.

Another way to run a single test is to click on a specific test case, and it will start executing.

If you want to run the test case, run the command ‘yarn cypress open’, which will open the Cypress Test Runner from where you can execute the test cases.

Execution

Let’s execute both test cases Searchwith_validAndinvalid_data.cy.js and subscriptionScenario.cy.js in non headless mode using UI by passing –ui in command.

You can also leverage the potential of Cypress E2E testing on the cloud and run Cypress test cases over real browsers and OS combinations. AI-powered test orchestration and execution platforms like LambdaTest offer a scalable Cypress cloud to perform automation testing across 40+ real desktop browsers.

Ready to elevate your Cypress automation skills? Join our specialized Cypress 101 certification program tailored for developers and testers seeking to expand their proficiency in Cypress test automation. Gain advanced insights, hone your abilities, and unlock a world of opportunities on your journey in test automation.

A comprehensive Exploratory Testing tutorial that covers what Exploratory Testing is, its importance, benefits, and how to perform it with real-time examples.

Conclusion

With Cypress CLI, you can run tests from the command line, which helps integrate tests into your CI/CD pipelines or run them in headless environments.

On the other hand, Cypress Test Runner offers a range of features that enhance the testing experience. It provides a real-time view of your application as tests run, allowing you to see each step executed and the application’s response. This helps in debugging and understanding the test flow.

Frequently Asked Questions (FAQs)

How to run Cypress cmd?

1.Navigate to Your Project Directory: Open your command prompt or terminal and navigate to the root directory of your Cypress project.

2. Run the Command: Use the following command to execute Cypress tests:

npx cypress open

This command will open the Cypress Test Runner, allowing you to select and run your test files interactively.

If you want to run tests in headless mode (without the interactive Test Runner UI), you can use:

npx cypress run

3.Select the Test File: In the Test Runner, select the test file you want to run from the list of available test files.

1. Run Tests: Click on the test file name to run the tests within that file.

What is a Cypress command?

The “Cypress command” typically refers to the command-line interface (CLI) commands provided by the Cypress testing framework. These commands allow you to interact with Cypress, run tests, manage configurations, and perform various testing-related tasks. Some common Cypress commands include:

• cypress open: Opens the Cypress Test Runner, providing an interactive interface to run and debug tests.

• cypress run: Executes tests in headless mode, which is suitable for automated testing and integration with CI/CD pipelines.

• cypress install: Installs the Cypress framework and its dependencies within your project.

• cypress verify: Checks the installed version of Cypress and its compatibility with your project.

• cypress –help: Displays the list of available commands and their descriptions.

• cypress –version: Displays the currently installed version of Cypress.

• cypress run –spec : Runs a specific test spec file.

• cypress run –record: Records test runs in the Cypress Dashboard for insights and videos.

• cypress run –parallel: Runs tests in parallel with a paid Cypress Dashboard plan.

These commands are executed in your terminal or command prompt and allow you to control various aspects of your testing workflow using the Cypress framework.