DEV Community: QAProEngineer

Handling Dynamic Preview URLs in Cypress Tests

QAProEngineer — Thu, 02 Jan 2025 21:55:36 +0000

When working with local development servers, a common challenge is dealing with dynamically generated preview URLs. Every time you start the server, a new URL is created, making it tricky to automate tests with tools like Cypress. In this article, I’ll walk you through two approaches to handle this issue and run Cypress tests seamlessly.
The Problem
When you run a local server (e.g., npm run start:local:test), it often generates a dynamic preview URL like https://random-preview-url.com. This URL changes every time the server starts, making it difficult to hardcode in your Cypress tests. Here’s how you can tackle this challenge.
**

Approach 1

Run the Server and Capture the Preview URL in Cypress
In this approach, we’ll start the server directly within the Cypress test suite and extract the preview URL from the server’s output.

`describe('Shopify App Preview Test', () => {
  let previewUrl;

  before(() => {
    // Run the server command and capture the output
    cy.exec('npm run start:local:test', { failOnNonZeroExit: true }).then((result) => {
      // Use a regex to extract the Preview URL from the command output
      const match = result.stdout.match(/Preview URL: (https:\/\/[^\s]+)/);
      if (match) {
        previewUrl = match[1]; // Extracted preview URL
      } else {
        throw new Error('Preview URL not found in server output');
      }
    });
  });

  it('should load the preview page', () => {
    // Visit the dynamically fetched preview URL
    cy.visit(previewUrl);

    // Add assertions to verify the preview page
    cy.contains('Expected content on the preview page').should('exist'); // Adjust as necessary
  });
});`

How It Works

The cy.exec() command runs the server and captures its output.
A regular expression extracts the preview URL from the server’s logs.
The extracted URL is used in the test to visit the page and perform assertions.

Approach 2

Use a Custom Script to Capture the Preview URL
In this approach, we’ll use a custom Bash script to start the server, extract the preview URL, and pass it to Cypress as an environment variable.

Step 1: Create a Bash Script
Save the following script as start-server.sh:

#!/bin/bash

# Run the server command in the background and capture the output to a log file
npm run start:local:test > server.log &

# Wait for a few seconds to allow the server to start and print the preview URL
sleep 5

# Extract the preview URL from the log file using `grep`
PREVIEW_URL=$(grep -oP 'Preview URL: \Khttps://[^\s]+' server.log)

# Export the preview URL as an environment variable
export CYPRESS_PREVIEW_URL=$PREVIEW_URL

# Run Cypress tests
npx cypress open

Step 2: Update Your Cypress Test

describe('Shopify App Preview Test', () => {
  let previewUrl;

  before(() => {
    // Get the preview URL from the environment variable
    previewUrl = Cypress.env('PREVIEW_URL');
  });

  it('should load the preview page', () => {
    // Visit the dynamically fetched preview URL
    cy.visit(previewUrl);

    // Add assertions as necessary
    cy.contains('Expected content on the preview page').should('exist');
  });
});

How It Works

The Bash script starts the server and logs its output.
It extracts the preview URL using grep and exports it as an environment variable.
Cypress reads the environment variable and uses it in the test.

Conclusion

Handling dynamic preview URLs in Cypress tests doesn’t have to be a headache. By using either of these approaches, you can automate your tests effectively, even when the URL changes every time you start the server. Give them a try and let me know which one works best for you!

Feel free to share your thoughts or questions in the comments below. Happy testing! 🚀

Extracting Links from Gmail Emails Using Node.js,Imap and Puppeteer

QAProEngineer — Thu, 02 Nov 2023 17:49:10 +0000

Introduction:

This article will walk you through the process of extracting links from Gmail emails using Node.js and Puppeteer. We'll explore how to set up an IMAP client to access your Gmail inbox, parse email content using the 'mailparser' library, and extract links from the email body using the 'cheerio' library. Additionally, we'll use Puppeteer to interact with and navigate to the extracted links within the emails.
Prerequisites:
Before we get started, make sure you have the following prerequisites in place:

A Gmail account: You will need a Gmail account from which you want to extract links.
App Password: To access your Gmail account programmatically, you should generate an App Password. This password is used in place of your regular Gmail password and is more secure for applications. We'll explain how to create an App Password in the article.
Node.js: Ensure you have Node.js installed on your computer. Getting Started: Creating an App Password

Gmail has a security feature that prevents the use of your regular password for less secure apps. To work around this, you can create an App Password. Here's how to do it:

Sign in to your Gmail account.
Go to your Google Account settings: - Click on your profile picture in the upper-right corner and select "Google Account." - In the left sidebar, click on "Security."
Under "Signing in to Google," click on "App passwords."
You may need to sign in again.
In the "App passwords" section, click "Select app" and choose "Mail" (for email) and "Other (Custom name)" for the device. Enter a custom name (e.g., "Node.js Email Extractor").
Click "Generate." You'll receive a 16-character app password. Make sure to copy it somewhere safe; you won't be able to see it again.

Setting Up the Code:
Now that you have an App Password, you can use it to access your Gmail account via IMAP and extract links from emails using Node.js. Here's a breakdown of the code you provided:

emailConfig: This object contains your Gmail account information, including the App Password you generated.
magicLinkSubject: This is the subject of the email you want to search for.
The code sets up an IMAP client, searches for emails with the specified subject, and then extracts and logs the email subject and body.
It uses 'cheerio' to parse the HTML content of the email and extract links within anchor tags. The links are stored in the links array and logged to the console.
Puppeteer is used to launch a browser, navigate to the first extracted link, and log the link.
Running the Code:

Make sure you have Node.js installed on your computer.
Install the required Node.js packages:

npm install imap mailparser cheerio puppeteer

Replace the emailConfig object's user and password properties with your Gmail address and the App Password you generated.
Run the script:

node your-script-filename.js

See below the full script:

const cheerio = require('cheerio');
const Imap = require('imap');
const { default: puppeteer} = require('puppeteer');
const simpleParser = require('mailparser').simpleParser;

const emailConfig = {
    user: 'test-email@gmail.com', // Replace with your Gmail email address
    password: 'generated-app-password', // Replace with your Gmail password
    host: 'imap.gmail.com',
    port: 993,
    tls: true,
    tlsOptions: {
        rejectUnauthorized: false, // For testing, you can disable certificate rejection
    },
    connectTimeout: 100000, // 60 seconds 
    authTimeout: 30000,
    debug: console.log,
};

const magicLinkSubject = 'Email subject example';

(async () => {
    // Set up an IMAP client
    const imap = new Imap(emailConfig);

    imap.once('ready', () => {
        imap.openBox('INBOX', true, (err) => {
            if (err) {
                console.error('Error opening mailbox', err);
                imap.end();
                return;
            }

            // Search for emails with the magic link subject
            imap.search([['SUBJECT', magicLinkSubject]], (err, results) => {
                if (err) throw err;

                const emailId = results[0]; // Assuming the first result is the correct email
                console.log('This is the email address: ' + emailId);
                const email = imap.fetch(emailId, { bodies: '' });

                email.on('message', (msg, seqno) => {
                    msg.on('body', (stream) => {
                        simpleParser(stream, async (err, mail) => {
                            if (err) throw err;

                            // Extract and log the email subject
                            const emailSubject = mail.text;
                            console.log('Email Subject:', emailSubject);
                            // Your code to extract and process the email content here
                            // Extract and log the email body
                            const emailBody = mail.html;
                            console.log('Email Body:', emailBody);
                            //Use cheerio to extract links 
                            const $ =cheerio.load(emailBody);
                            const links=[];
                            $('a').each((index,element)=>{
                                links.push($(element).attr('href'));

                            });
                            console.log('Extracted Links', links);
                            const browser =await puppeteer.launch({headless: false});
                            const page = await browser.newPage();
                            await page.goto(links[0]);
                            console.log('this is the first link'+ links[0]);

                        });
                    });
                });
            });
        });
    });

    imap.connect();

    // Handle errors and edge cases as needed
})();

How to Deploy Puppeteer with AWS Lambda

QAProEngineer — Wed, 25 Oct 2023 15:58:46 +0000

Prerequisites

Before we dive into the details, make sure you have the following in place:

An AWS account with access to AWS Lambda and other necessary services.
A Node.js project where you can deploy your Lambda function.
Familiarity with Puppeteer and basic AWS Lambda concepts.

Setting up the Environment

The first step is to set up your development environment. In your Node.js project, you'll need to install the **chrome-aws-lambda **package. This package contains a headless Chromium binary optimized for AWS Lambda environments.
npm install chrome-aws-lambda
Once the package is installed, you can create a Lambda function to run your Puppeteer code.

Creating the AWS Lambda Function

For this example, we'll create a Lambda function that navigates to a webpage and returns its title. Here's a sample Lambda function:

const chromium = require('chrome-aws-lambda');

exports.handler = async (event, context) => {
  let result = null;
  let browser = null;

  try {
    browser = await chromium.puppeteer.launch({
      args: chromium.args,
      defaultViewport: chromium.defaultViewport,
      executablePath: await chromium.executablePath,
      headless: chromium.headless,
      ignoreHTTPSErrors: true,
    });

    let page = await browser.newPage();

    await page.goto(event.url || 'https://example.com');

    result = await page.title();
  } catch (error) {
    console.error(error);
  } finally {
    if (browser !== null) {
      await browser.close();
    }
  }

  return result;
};

Let's break down the key parts of this Lambda function:

We import the chrome-aws-lambda package, which provides an optimized Chromium binary for AWS Lambda.
Inside the Lambda handler, we launch a Puppeteer browser using the chromium.puppeteer.launch method. We pass various options, such as command-line arguments and the path to the Chromium binary.

1- We create a new page, navigate to a URL (you can specify the URL as an event parameter), and retrieve the page title.
2-If any errors occur during the process, we log them to the console.
3- Finally, we return the result, which is the page title in this example.

Deploying the Lambda Function

To deploy the Lambda function, you can use the AWS Management Console, AWS CLI, or a tool like the Serverless Framework. Here, we'll use the AWS CLI as an example:
Ensure you have the AWS CLI configured with the necessary permissions.
Create a deployment package by packaging your Node.js code and its dependencies. In your project directory, run the following command:

zip -r function.zip node_modules/ your-function.js

1-Replace your-function.js with the name of your Lambda function file.
Create the Lambda function using the AWS CLI. Replace with the appropriate IAM role ARN that gives Lambda permissions to access other AWS resources.

aws lambda create-function --function-name YourFunctionName \
  --zip-file fileb://function.zip \
  --handler your-function.handler \
  --runtime nodejs14.x \
  --role <ROLE_ARN>

2-Invoke your Lambda function using the AWS CLI or any other method you prefer:

aws lambda invoke --function-name YourFunctionName output.txt

Replace YourFunctionName with the name of your Lambda function. You should see the title of the webpage in the output.txt file.

Conclusion

Running Puppeteer in an AWS Lambda function can be a powerful way to automate browser tasks on a serverless infrastructure. With chrome-aws-lambda, you can use an optimized Chromium binary, reducing the cold start time and improving the overall performance of your Lambda functions. This can be useful for various use cases, including web scraping, automated testing, and generating PDFs from web pages. Just keep in mind that AWS Lambda has some limitations, such as execution time limits and memory restrictions, so it's important to design your functions accordingly.

Step-by-Step Guide to Setting Up Sauce Labs CI/CD with GitHub Actions for mobile App testing

QAProEngineer — Tue, 26 Sep 2023 15:23:40 +0000

Introduction

Sauce Labs is a cloud-based platform for automated testing of web and mobile applications. In this guide, we'll walk through the steps to set up a CI/CD (Continuous Integration/Continuous Deployment) pipeline using GitHub Actions to perform end-to-end (E2E) testing of a mobile app with Sauce Labs. The GitHub Actions workflow is defined in a YAML file (.github/workflows/saucelabs.yml), and it consists of several steps.

Prerequisites

Before getting started, make sure you have the following prerequisites in place:

A GitHub repository containing your mobile app source code.
A Sauce Labs account with SAUCE_USERNAME and SAUCE_ACCESS_KEY credentials.
The mobile app file (APK in this example) you want to test.

GitHub Repository Setup

Create a .github/workflows directory in your GitHub repository if it doesn't already exist.
Inside the .github/workflows directory, create a YAML file (e.g., saucelabs.yml) to define your CI/CD workflow.
Copy and paste the following YAML code into your saucelabs.yml file:

name: Sauce CI:  app E2E testing

on:
  - push

env:
  SAUCE_USERNAME: ${{ secrets.SAUCE_USERNAME }}
  SAUCE_ACCESS_KEY: ${{ secrets.SAUCE_ACCESS_KEY }}

jobs:
  build:
    runs-on: macos-latest

    steps:
      - name: Checkout Repository
        uses: actions/checkout@v2

      - name: Set Up Node.js
        uses: actions/setup-node@v2
        with:
          node-version: 16.13.0

      - name: Upload App to Sauce Labs
        run: |
          SAUCE_USERNAME=$SAUCE_USERNAME
          SAUCE_ACCESS_KEY=$SAUCE_ACCESS_KEY
          APP_FILE_PATH=$(pwd)/app-upload/app-release-2.0.5.apk

          response=$(curl -u "$SAUCE_USERNAME:$SAUCE_ACCESS_KEY" --location \
          --request POST 'https://api.us-west-1.saucelabs.com/v1/storage/upload' \
          --form "payload=@$APP_FILE_PATH" \
          --form 'name="app-x86-release.apk"' \
          --form 'description="Android Test App v3"')

          # Extract the file ID from the response
          file_id=$(echo "$response" | jq -r '.item.id')

          # Print the file_id value
          echo "File ID to be deleted: $file_id"

          # Store the file_id in a file
          echo "$file_id" > file_id.txt

      - name: Change to e2e/testing directory
        run: cd e2e/testing

      - name: Install Dependencies
        run: |
          cd e2e/testing
          npm install
          npm run test:android:sauce:emu 
        continue-on-error: true  # Continue even if tests fail

      - name: Delete App from Sauce Labs
        run: |
          SAUCE_USERNAME=$SAUCE_USERNAME
          SAUCE_ACCESS_KEY=$SAUCE_ACCESS_KEY

          # Read the file_id from the stored file
          file_id=$(cat file_id.txt)

          # Print the file_id value
          echo "File ID to be deleted: $file_id"

          # Add a delay of 10 seconds
          sleep 10

          curl -u "$SAUCE_USERNAME:$SAUCE_ACCESS_KEY" --location \
          --request DELETE "https://api.us-west-1.saucelabs.com/v1/storage/files/$file_id"

      - name: Upload Logs on Failure
        if: failure()
        uses: actions/upload-artifact@v2
        with:
          name: logs
          path: logs

Let's break down what each step in the workflow does:

Checkout Repository: This step checks out the code from your GitHub repository.
Set Up Node.js: Installs Node.js with the specified version (16.13.0 in this case) on the GitHub runner.
Upload App to Sauce Labs: Uploads your mobile app (APK file) to Sauce Labs storage and stores the file ID in a text file for future reference.
Change to e2e/testing directory: Navigates to the directory where your end-to-end tests are located.
Install Dependencies: Installs project dependencies and runs the E2E tests for the Android app on Sauce Labs Android emulators. The continue-on-error flag is set to true to allow the workflow to continue even if the tests fail.
Delete App from Sauce Labs: Deletes the app from Sauce Labs storage using the file ID stored earlier. This is optional but helps clean up storage after the tests.
Upload Logs on Failure: If any step in the workflow fails (e.g., test failures), this step uploads the logs as artifacts for debugging.

GitHub Secrets

To securely store your Sauce Labs credentials (SAUCE_USERNAME and SAUCE_ACCESS_KEY), you should add them as secrets in your GitHub repository. To add secrets, follow these steps:

Go to your GitHub repository.
Click on "Settings" > "Secrets" > "New repository secret."
Add a secret with the name SAUCE_USERNAME and set its value to your Sauce Labs username.
Add another secret with the name SAUCE_ACCESS_KEY and set its value to your Sauce Labs access key.

Conclusion

With this GitHub Actions workflow, you can automate the E2E testing of your mobile app using Sauce Labs. When you push changes to your repository, GitHub Actions will trigger the workflow, and you'll receive test results and logs in case of failures. This setup helps ensure the quality and reliability of your mobile application through automated testing with Sauce Labs.

Remember to customize the workflow according to your specific project requirements, such as adjusting the Node.js version or the paths to your app and test files.

Analyzing data accurancy with Puppeteer and Axios( example: book prices )

QAProEngineer — Wed, 13 Sep 2023 19:30:34 +0000

const puppeteer = require("puppeteer");
const axios = require("axios");

const main = async () => {
  // Launch a new browser instance
  const browser = await puppeteer.launch({
    headless: false,
    defaultViewport: null,
  });

  // Create a new page
  const page = await browser.newPage();

  // Navigate to the book product details page
  const baseUrl = "https://www.example.com/book-top-100?p=";

  // define the number of pages to iterate through.
  const pageCount = 2;

  // Create a new array for the results
  const results = [];

  //Iterate through each page
  for (let i = 1; i < pageCount; i++) {
    //construct the URL for the current page
    const currentUrl = `${baseUrl}${i}`;
    //navigate to the page
    await page.goto(currentUrl);

    // get all the product listing urls
    const urls = await page.$$eval("li.item.product > div > a", (links) =>
      links.map((link) => link.href)
    );

    for (const url of urls) {
      await page.goto(url);

      // Get all the swatches for the book
      const swatchElements = await page.$$("p.label");

      // Loop through each swatch and click on it to show the price and SKU
      for (let swatchElement of swatchElements) {
        await swatchElement.click();

        // Extract the price and SKU
        const sku = await page.$eval(".col.data.isbn_13", (elem) =>
          elem.textContent.trim()
        ); // extract the SKU for each selection of format.

        let priceText;
        const swatchElementType = await swatchElement.evaluate((el) =>
          el.textContent.trim()
        );
        if (
          swatchElementType.includes("Audiobook") ||
          swatchElementType.includes("eBook")
        ) {
          priceText = await page.$$eval(
            ".price-swatch span.price, .normal-price",
            (prices) => prices.map((price) => price.textContent.trim())
          );
        } else if (
          swatchElementType.includes("Paperback") ||
          swatchElementType.includes("Hardcover")
        ) {
          priceText = await page.$$eval(
            "p.old-price, span.old-price",
            (prices) => prices.map((price) => price.textContent.trim())
          );
        }

        const price = priceText.flatMap((price) =>
          price.split("$").filter(Boolean)
        ); // extract the numerical price values for this product

        // Make an API call to API to extract the price values for this product then compare them against the web prices.
        const api_book = `https://api.example.com/products/${sku}`;
        const response = await axios.get(api_book);

        const apiPrice = response.data[0].price_amount;
        const webPrice = Number.parseFloat(price);

        console.log(
          "this the api price=" + apiPrice + " and " + "web price=" + price
        );
        if (webPrice == apiPrice) {
          console.log(`Price for SKU ${sku} matches in web and api price`);
        } else {
          console.log(`Price for SKU ${sku} does not match in web and API`);
          // Add the results to the array
          results.push({
            price,
            apiPrice,
            sku,
          });
        }
      }
    } // end url loop
  }
  //Append The results to a string with new lines
  const resultString = results
    .map(
      (result) =>
        `Price: ${result.price}\nAPI Price: ${result.apiPrice}\nSKU: ${result.sku}\n`
    )
    .join("\n");
  // Close the browser instance
  await browser.close();

  // Write the results to a JSON file
  const fs = require("fs");
  fs.writeFileSync("book-prices-details.json", JSON.stringify(resultString));
};

main();

Introduction:
In today's digital age, it's essential for businesses to stay competitive, especially when it comes to pricing their products. One way to do this is by regularly comparing your product prices with those of your competitors. This blog post will explain the purpose of a Puppeteer script that automates the process of extracting book prices from a website and comparing them with prices from an API.

Puppeteer and Axios:
Puppeteer is a headless browser automation tool that allows you to control and interact with web pages programmatically. Axios is a popular JavaScript library for making HTTP requests. Together, they provide a powerful toolset for web scraping and data extraction.

Script Overview:
The provided Puppeteer script has several key objectives:

1.Launch a headless web browser using Puppeteer.
2.Navigate to a specific website that lists the top 100 books.
3.Iterate through multiple pages of book listings.
4.Extract the URLs of individual book product pages.
5.For each book product page, extract the price and SKU information for different book formats (e.g., Audiobook, eBook, Paperback, Hardcover).
6.Make API calls to a price comparison service to fetch the correct prices for the books.
7.Compare the web prices with the API prices to identify any discrepancies.
8.Store the results in a structured format (JSON file).

Conclusion:
This Puppeteer and Axios script automates the process of comparing book prices between a website and an API, helping businesses make informed pricing decisions. It demonstrates the power of web scraping and data extraction with Puppeteer and the ease of making API requests with Axios. By regularly running such scripts, businesses can stay competitive and ensure their pricing remains competitive in the market.

Write a Playwright test using AWS Lambda

QAProEngineer — Mon, 11 Sep 2023 16:08:26 +0000

Create a Playwright Test Script:

Write your Playwright test script in a Node.js environment. Here's a simple example using Playwright to open a webpage and take a screenshot:
`const { chromium } = require('playwright');

exports.handler = async (event) => {
let browser;
try {
browser = await chromium.launch();
const context = await browser.newContext();
const page = await context.newPage();

await page.goto('https://example.com');
await page.screenshot({ path: '/tmp/example.png' });

await browser.close();

return {
  statusCode: 200,
  body: JSON.stringify({ message: 'Screenshot taken!' }),
};

} catch (error) {
console.error('Error:', error);

return {
  statusCode: 500,
  body: JSON.stringify({ error: 'Internal Server Error' }),
};

} finally {
if (browser) await browser.close();
}
};`

Package Dependencies:

Since AWS Lambda has limited space for dependencies, you should use a tool like serverless or AWS SAM to package your Lambda function along with its dependencies. Ensure that you package Playwright and any other required Node.js modules.

3.Deploy to AWS Lambda:

Use AWS CLI or a deployment tool like serverless or AWS SAM to deploy your Lambda function to AWS.

4.Configure Lambda Function:

In the AWS Lambda console, set up your function. You'll need to configure the following:

** Runtime:** Node.js (Choose the version according to your script's requirements).
** Handler:** The name of your Lambda function and the exported function (e.g., filename.handler).
Memory: Set memory as per your requirements.
** Timeout:** Set the timeout duration for your function. Ensure it's long enough to complete your Playwright test.
** Execution Role:** Create an IAM role that allows the Lambda function to execute other AWS services and resources as needed.

5.Create an API Gateway (Optional):

If you want to trigger your Lambda function via an HTTP request, you can create an API Gateway and configure it to invoke your Lambda function.

6.Testing the Lambda Function:

You can now test your Lambda function by invoking it either through the AWS Lambda console or by making an HTTP request if you set up API Gateway.