DEV Community: Nathan Araújo

Streamline Your Test Automation with Azure Test Track VS Code Extension

Nathan Araújo — Thu, 23 Oct 2025 13:08:58 +0000

Why This Process Matters: Microsoft's Automation Status Design

The Hidden Challenge with Azure DevOps Automation Status

Here's something most QAs don't realize: You cannot manually set the "Automation Status" field to "Automated" in Azure DevOps. The dropdown only shows "Not Automated" and "Planned" options. So how do test cases get the "Automated" status? Here's the secret:

Microsoft's Intelligent Design Philosophy

Microsoft designed Azure DevOps with a specific workflow in mind:

Associated Automation Tab = Source of Truth
- The "Associated Automation" tab should contain the actual automation details
- This tab represents the real connection between test cases and automated tests
Automation Status = Calculated Field
- The "Automation Status" field is designed to be automatically triggered
- When you populate the "Associated Automation" tab, Microsoft's internal triggers update the status
- This ensures the status reflects actual automation, not manual changes
Why Manual Updates Don't Work
- The "Automation Status" dropdown only has "Not Automated" and "Planned"
- "Automated" is not a selectable option - it only appears when triggered by the system
- You cannot manually mark a test case as "Automated" even if you want to
- The only way to get "Automated" status is through proper automation association

The Real Challenge

How do you actually get a test case to show "Automated" status?

❌ Try to select "Automated": Not possible - option doesn't exist in dropdown
❌ Leave as "Not Automated": Doesn't reflect reality of your automation efforts
✅ Populate Associated Automation: The ONLY way to trigger "Automated" status

Microsoft's Recommended Workflow

📝 Write Automated Test
    ↓
🔗 Populate Associated Automation Tab  
    ↓
⚙️  Microsoft Triggers Update Status
    ↓ 
✅ Automation Status = 'Automated'
    ↓
📋 Full Traceability Maintained

What This Extension Solves

By automatically populating the Associated Automation tab, we:

✅ Follow Microsoft's intended workflow
✅ Trigger the automated status update correctly
✅ Maintain complete traceability from code to test case
✅ Ensure accuracy - status reflects real automation
✅ Enable verification - anyone can see the actual test details

Key Insight: This isn't just about saving time - it's about following Microsoft's design pattern to ensure your automation status is trustworthy, verifiable, and properly integrated with Azure DevOps' automation tracking system.

The Problem Every QA Faces

As QA professionals, we often struggle with:

Manual test case association in Azure DevOps taking hours
Lost traceability between automated tests and test cases
Outdated automation status in ADO that doesn't reflect reality
Time-consuming manual updates every time we add new tests

What if there was a better way?

Introducing Azure Test Track Extension

The Azure Test Track VS Code Extension revolutionizes how you manage test automation by automatically linking your automated tests to Azure DevOps Test Cases with just a simple comment.

What It Does

Automatically associates automated tests with ADO Test Cases
Updates automation status to "Automated" in Azure DevOps
Populates Associated Automation tab with all relevant details
Supports multiple languages: JavaScript, TypeScript, Python, and Gherkin
Visual indicators show which tests are linked vs unlinked

How It Works

Step 1: Add ADO_IDs Comment

Simply add a comment above your test with the test case IDs:

// ADO_IDs: TC_1234, TC_5678
test('User can login with valid credentials', async () => {
  // Your test code here
});

Step 2: Run the Association Command

Use VS Code Command Palette (Ctrl+Shift+P) and run:

Associate IDs from Comments

Step 3: See the Magic in Azure DevOps!

See It In Action

Before: Manual Test Case Management

Traditional manual process of updating test cases one by one

Azure DevOps Integration Results

1. Associated Automation Tab - Populated Automatically

The Associated Automation tab is automatically filled with test details

2. Automation Status Changed to "Automated"

VS Code Integration

1. Visual Indicators in Your Code

Green checkmarks show associated tests, red X shows unassociated tests

Quick Start Guide

1. Installation

# Install from VS Code Extensions Marketplace
# Search for "Azure Test Track"

2. Setup Environment Variables Globally

# Required ADO configuration
ADO_ORGANIZATION=your-organization
ADO_PROJECT=your-project
ADO_PERSONAL_ACCESS_TOKEN=your-pat-token
ADO_COMPANY_EMAIL=your-email@company.com

3. Start Using

For JavaScript/TypeScript:

// ADO_IDs: TC_1001
test('API returns user data correctly', () => {
  // Test implementation
});

For Python:

# ADO_IDs: TC_2001
def test_user_authentication():
    # Test implementation
    pass

For Gherkin (.feature files):

# ADO_IDs: TC_3001
Scenario: User can complete checkout process
  Given user has items in cart
  When user proceeds to checkout
  Then order should be created successfully

Pro Tips for QA Teams

1. Batch Processing

Associate all tests in a file at once:

# Use "Associate IDs from Comments" command
# Processes entire file automatically

2. Visual Verification

Use the decoration feature to see association status:

✅ Green: Test is associated with ADO
❌ Red: Test needs association

3. Multiple Test Cases

One test can cover multiple test cases:

// ADO_IDs: TC_1001, TC_1002, TC_1003
test('Complete user registration flow', () => {
  // Covers validation, creation, and confirmation
});

Real Impact on QA Productivity

Quality Improvements

✅ 100% accurate test case associations
✅ Real-time sync between code and ADO
✅ Zero manual errors in test case updates
✅ Complete traceability from requirement to test

Enterprise Ready

Security & Compliance

✅ Uses Azure DevOps PAT tokens (secure authentication)
✅ Respects organization permissions
✅ No data stored outside your environment
✅ Works with enterprise Azure DevOps instances

Team Collaboration

✅ Consistent test case association across team
✅ Standardized ADO_IDs format
✅ Visual indicators for code reviews
✅ Automated documentation updates

Need Help?

📖 Documentation: README.md
💡 Examples: Code Examples
🐛 Issues: GitHub Repository
📧 Contact: Support team

🔗 Quick Links

[Playwright] Element is outside of the viewport" in Playwright — why it happens and how to fix it

Nathan Araújo — Fri, 26 Sep 2025 11:32:39 +0000

The problem

You have a checkbox in the DOM, for example, with id="AcceptedTermsOfUse", that is outside of the visible area of the page (viewport), for example with very large negative x coordinates like x = -9459.

When you try to run:

await page.locator('#AcceptedTermsOfUse').click();

Playwright throws an error like:

Error: locator.click: Test timeout ...

“element is outside of the viewport”

or
"Call log:
- waiting for locator('input[type="checkbox"][name="AcceptedTermsOfUse"]')
- locator resolved to "

This means: the element exists, it is “visible” in terms of CSS (not display:none, not visibility:hidden), but it is not in a position where a user could physically click it, because it’s positioned or hidden off‑screen.

Why this happens

Common causes:

Off‑screen styling

The input is styled with position: absolute (or similar), using left: -xxxxx or transform that pushes it outside the viewport.

Styled label + hidden input

Many UI frameworks hide the real off-screen and use only a label plus pseudo-elements (::before, ::after) to render the visible checkbox square. This allows them to style a “fancy checkbox” without relying on the browser’s native styling. The user clicks on the label, which propagates the click to the hidden input. So the input is technically visible (not display:none), but positioned outside the viewport so the user never interacts with it directly.

Playwright actionability checks

Before performing locator.click(), Playwright ensures:

the element is visible
enabled
stable (not moving/animating)
within the viewport or can be scrolled into it

If any of these fail, Playwright won’t click.

Known tricky cases

Sometimes elements are technically visible but still unreachable due to layout/CSS. This is common with checkboxes hidden for custom styling.

Real example

In this case, the checkbox was:

Visible in CSS terms
Enabled
But with boundingBox().x = -9459 — far to the left
Absolutely positioned
No scrollable parent to bring it into view

So Playwright couldn’t click because it wasn’t physically actionable.

Possible solution

1. Use `page.evaluate` with `element.click()`

await page.evaluate(() => {
  const checkbox = document.getElementById("AcceptedTermsOfUse");
  if (checkbox) checkbox.click();
});

This works even if the input is off‑screen, because it fires a DOM click programmatically.

2. Manually set `checked` and dispatch events

await page.evaluate(() => {
  const checkbox = document.getElementById("AcceptedTermsOfUse") as HTMLInputElement;
  if (checkbox) {
    checkbox.checked = true;
    checkbox.dispatchEvent(new Event('change', { bubbles: true }));
  }
});

Forces the state and triggers a change event.

Investigation

You can investigate this behavior directly in the browser console. For example, open DevTools and run:

document.getElementById("AcceptedTermsOfUse").click();

You’ll notice the checkbox gets checked and any attached event listeners fire — even if the element is off-screen.

Why `document.getElementById("AcceptedTermsOfUse").click()` works off-screen, but Playwright’s `locator.click()` does not?

When you call element.click() in the browser (either in the console or inside page.evaluate()), the browser’s DOM engine executes HTMLElement.prototype.click().

This method programmatically triggers all associated events (mousedown, mouseup, click, input, change, depending on the element) and updates the element state. Importantly:

The browser does not require the element to be inside the viewport.
The DOM allows simulating a click even if the element is positioned off-screen.

Why Playwright behaves differently

Playwright aims to simulate real user interaction, not just fire DOM events. When you call:

await page.locator('#AcceptedTermsOfUse').click();

Playwright performs actionability checks:

The element must be visible, enabled, stable (not moving), and not covered by other elements.
The element must be within the viewport, or Playwright will attempt to scroll it into view.
If Playwright cannot make the element actionable, it throws: "element is outside of the viewport".

These checks exist so tests mimic how a real user would interact with the page — for example, preventing clicks through invisible or off-screen elements.

Exploring Azure Functions for Synthetic Monitoring with Playwright: A Complete Guide - Part 5

Nathan Araújo — Mon, 15 Sep 2025 15:37:18 +0000

Over the past few days, I’ve been diving into using Azure Functions to implement synthetic monitoring with Playwright. The goal was simple: run browser automation tests every 30 minutes or 1 hour to monitor critical system functionalities. However, as with many cloud adventures, there were some surprises along the way.

1. Choosing the Right Plan

The first dilemma was: Consumption Plan or Premium Plan?

The Consumption Plan is cheap but does not support running browsers like Playwright because of strict sandbox limitations.

The Premium Plan eliminates cold starts, offers dedicated CPU/RAM, and allows running browsers — essential for automation tasks.

Within the Premium Plan, there are different SKUs: EP1, EP2, and EP3. For a proof of concept (POC) running tests every 30 minutes or 1 hour, EP1 (1 vCPU / 3.5 GB RAM) is enough and much cheaper (~$130–150/month). Heavier workloads require EP2 or EP3 with more CPU and memory.

I opted for EP1. This ensures Playwright runs stably, even when opening headless browsers.

2. Deployment Structure and “No Functions Found”

Initially, I deployed my Function App via .zip using Azure Pipeline and Release, including dist/, host.json, and other project folders. However, the logs showed:

0 functions found

I discovered that Azure Functions requires the function folder to be at the root of the deployment or correctly configured. In modern Functions (v4, JavaScript/TypeScript), each function file must use @azure/functions and host.json needs to be at the root.

To maintain my project structure (dist/functions, dist/support, dist/tests), the solution was to point the deployment to the root of dist and ensure host.json and .js function files were correctly located.

3. Playwright and Browsers

One of the biggest challenges was running Playwright on Azure.

Chromium works fine ✅

Firefox and WebKit fail ❌

Typical error:

Error: browserType.launch: spawn UNKNOWN

This happens because the Azure Functions Windows sandbox doesn’t provide all system dependencies Firefox/WebKit require.

Possible Solutions:

Use Chromium only – works reliably and is sufficient for synthetic monitoring.
Deploy via Linux Function App + --with-deps – installs necessary system libraries.
Custom Container – a Docker image based on mcr.microsoft.com/playwright comes with Chromium, Firefox, and WebKit pre-installed with dependencies. This is ideal for production.

4. Environment Variables and Paths

To make Playwright locate the browser, I had to set:

PLAYWRIGHT_BROWSERS_PATH = 0

This ensures the Azure Function looks for browsers in the correct location in node_modules. If you install browsers in the pipeline with npx playwright install chromium, the node_modules folder must be included all browsers that you need and with PLAYWRIGHT_BROWSERS_PATH = 0 Playwright will look at for the right location.

Conclusion

After going through all of this, I finally succeeded in getting the Azure Function TimerTrigger to run reliably. The execution traces were sent to Application Insights, allowing me to monitor each run in real time. Additionally, the errors that occurred during the tests helped me realize that the reports were being stored in Azure Blob Storage, providing a reliable way to analyze test results and gain insights for further improvements.

Additional Resources

Exploring Azure Functions for Synthetic Monitoring with Playwright: A Complete Guide - Part 4

Nathan Araújo — Fri, 12 Sep 2025 13:15:01 +0000

Azure Functions Deployment

Deploy your Synthetic Monitoring solution to Azure in 4 simple phases.

Phase 1: Create Azure Resources

1. Create Function App

Azure Portal → Create Resource → Function App
Configure:
- Name: synthetic-monitoring-func-prod
- Runtime: Node.js 18
- Plan: Functions Premium (production)
- Storage: Create new
- Application Insights: Enable

2. Create Storage Account (for artifacts)

Azure Portal → Storage Accounts → Create
Configure:
- Name: syntheticartifacts[suffix]
- Performance: Standard
- Create container: test-artifacts

3. Get Connection Strings

Application Insights: Properties → Connection String
Storage Account: Access Keys → Connection String

Phase 2: Configure Function App

Add these variables in Function App → Settings → Environment Variables:

APPLICATIONINSIGHTS_CONNECTION_STRING=[App Insights Connection String]
AZURE_STORAGE_CONNECTION_STRING=[Storage Account Connection String]
BLOB_CONTAINER_NAME=test-artifacts
baseUrl=https://your-target-application.com
SYNTHETIC_MONITOR_SCHEDULE=0 */5 * * * *
... others env variables

Phase 3: Setup Azure DevOps Pipeline

1. Create Build Pipeline

Create azure-pipelines.yml in your repo:

trigger:
  - main

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: NodeTool@0
  inputs:
    versionSpec: '18.x'

- script: npm ci
  displayName: 'Install dependencies'

- script: npx playwright install --with-deps chromium
  displayName: 'Install Playwright'

- script: npm run build
  displayName: 'Build TypeScript'

- task: ArchiveFiles@2
  inputs:
    archiveFile: '$(Build.ArtifactStagingDirectory)/function-app.zip'
    includeRootFolder: false

- task: PublishBuildArtifacts@1
  inputs:
    artifactName: 'function-app'

Or classic mode

2. Create Release Pipeline

Pipelines → Releases → New Pipeline
Add Artifact: Link to build pipeline
Add Stage: Deploy to Function App
Configure Task:
- Azure Function App Deploy
- App Service name: synthetic-monitoring-func-prod
- Package: $(System.DefaultWorkingDirectory)/_[BuildName]/function-app/function-app.zip

Phase 3: Deploy and Monitor

1. Deploy

Commit code → Build runs automatically
Release pipeline → Deploy to Function App
Verify function is running in Azure Portal

2. Monitor

Application Insights → Live Metrics (real-time monitoring)
Function App → Monitoring → Log Stream (logs in real time)
Function App → Functions → syntheticMonitorFunctionName (execution history)
Storage Account → test-artifacts (uploaded reports on failures)

Quick Checklist

[ ] Function App created with Application Insights
[ ] Storage Account created with container
[ ] Connection strings configured in Function App
[ ] Build pipeline runs successfully
[ ] Release pipeline deploys to Function App
[ ] Function executes and sends telemetry
[ ] Test artifacts upload on failures

Additional Resources

Exploring Azure Functions for Synthetic Monitoring with Playwright: A Complete Guide - Part 3

Nathan Araújo — Thu, 11 Sep 2025 12:54:10 +0000

Running Synthetic Monitoring with Azurite and Local Testing

Developing and testing synthetic monitoring solutions locally is crucial for rapid development cycles and debugging. This article demonstrates how to run your Azure Functions + Playwright synthetic monitoring solution locally using Azurite (Azure Storage Emulator) for blob storage and local Application Insights configuration.

This setup allows you to:

Test your monitoring solution without Azure costs
Debug issues in a controlled environment
Validate changes before deploying to production
Develop offline or with limited internet connectivity

Local Development Architecture

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Azure Timer   │    │  Azure Function │    │   Playwright    │
│   Trigger       │───▶│   (Runtime)     │───▶│   Test Runner   │
│  (Azurite)      │    │                 │    │                 │
└─────────────────┘    └─────────────────┘    └─────────────────┘
                                                        │
                                                        ▼
                                              ┌─────────────────┐
                                              │  Test Results   │
                                              │   Processing    │
                                              └─────────────────┘
                                                        │
                                              ┌─────────┴─────────┐
                                              ▼                   ▼
                                     ┌─────────────────┐ ┌─────────────────┐
                                     │ Application     │ │   Azure Blob    │
                                     │ Insights        │ │   Storage       │
                                     │ (Telemetry)     │ │ (Artifacts)     │
                                     └─────────────────┘ └─────────────────┘

Prerequisites

Before starting, ensure you have installed:

Node.js 18+
Azure Functions Core Tools v4
Azurite (Azure Storage Emulator)
Azure Storage Explorer (optional, for GUI access)

# Install Azure Functions Core Tools
npm install -g azure-functions-core-tools@4 --unsafe-perm true

# Install Azurite globally
npm install -g azurite

# Or install as dev dependency in your project
npm install -D azurite

Setting Up Azurite for Local Blob Storage

1. Azurite Installation and Configuration

Azurite emulates Azure Blob Storage, Queue Storage, and Table Storage locally.

Global Installation Method:

# Install globally
npm install -g azurite

# Start Azurite with default settings
azurite --silent --location ./azurite --debug ./azurite/debug.log

2. Azurite Configuration Options

Create an azurite configuration for your project:

# Start Azurite with custom configuration
azurite \
  --blobHost 127.0.0.1 \
  --blobPort 10000 \
  --queueHost 127.0.0.1 \
  --queuePort 10001 \
  --tableHost 127.0.0.1 \
  --tablePort 10002 \
  --location ./azurite \
  --debug ./azurite/debug.log \
  --silent

Configuration Parameters:

--location: Directory where Azurite stores data
--debug: Debug log file location
--silent: Reduces console output
--blobPort: Port for blob storage (default: 10000)
--loose: Enables loose mode for compatibility

Local Environment Configuration

1. Local Settings Configuration

Create or update your local.settings.json file:

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;",
    "FUNCTIONS_WORKER_RUNTIME": "node",
    "AZURE_STORAGE_CONNECTION_STRING": "your-storage-account-connection-string",
    "BLOB_CONTAINER_NAME": "test-artifacts",
    "APPLICATIONINSIGHTS_CONNECTION_STRING": "your-appinsights-connection-string",
    "baseUrl": "https://example.com",
    "SYNTHETIC_MONITOR_SCHEDULE": "0 */5 * * * *"
  }
}

Key Configuration Points:

AzureWebJobsStorage: Required by Azure Functions runtime
AZURE_STORAGE_CONNECTION_STRING: Used by your blob storage functions
APPLICATIONINSIGHTS_CONNECTION_STRIN: Used by your appInsights functions to report the test status
BLOB_CONTAINER_NAME: Container for storing test artifacts
SYNTHETIC_MONITOR_SCHEDULE: Set to run every minute for local testing

2. Complete Local Development Workflow

Terminal 1 - Start Azurite:

# manually:
azurite --silent --location ./azurite --debug ./azurite/debug.log

Terminal 2 - Build and Start Azure Functions:

# Build TypeScript
npm run build

# Start Azure Functions locally
func start --typescript

Terminal 3 - Run Tests Manually (Optional):

# Run Playwright tests directly
npx playwright test

Conclusion

Setting up a local development environment with Azurite enables rapid development and testing of your synthetic monitoring solution. This approach provides:

Fast Development Cycles: No need to deploy to Azure for every change
Cost Savings: No Azure storage costs during development
Debugging Capabilities: Full control over the local environment

The combination of Azurite for storage emulation and local Azure Functions runtime creates a robust development environment that closely mirrors production behavior while providing the flexibility needed for effective development and testing.

Additional Resources

Exploring Azure Functions for Synthetic Monitoring with Playwright: A Complete Guide - Part 2

Nathan Araújo — Wed, 10 Sep 2025 19:23:59 +0000

Introduction

In the first article, we explored how to build a synthetic monitoring solution with Azure Functions and Playwright. Now, we'll dive deep into the reporting mechanism - the crucial component that transforms test results into actionable insights by sending telemetry to Application Insights and storing test artifacts in Azure Blob Storage.

This article focuses on the Test Results Processing layer of our architecture, explaining how the custom Playwright reporter works, when and how it sends data to Azure services, and best practices for monitoring and troubleshooting.

The Reporting Architecture

┌─────────────────┐
│   Playwright    │
│   Test Runner   │
│                 │
└─────────────────┘
          │
          ▼
┌─────────────────┐
│  Test Results   │ ◄── Custom Reporter Implementation
│   Processing    │
└─────────────────┘
          │
┌─────────┴─────────┐
▼                   ▼
┌─────────────────┐ ┌─────────────────┐
│ Application     │ │   Azure Blob    │
│ Insights        │ │   Storage       │
│ (Telemetry)     │ │ (Artifacts)     │
└─────────────────┘ └─────────────────┘

Understanding Playwright Reporters

Playwright reporters are plugins that process test results during and after test execution. Our custom reporter implements the Reporter interface with key lifecycle methods:

onTestEnd() - Called when each individual test completes
onEnd() - Called when the entire test run finishes

Configuring the Custom Reporter in Playwright

1. Playwright Configuration Setup

The custom reporter is integrated into the Playwright configuration file (playwright.config.ts). Here's how it's configured:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  // ... other configuration options

  reporter: [
    ['html', { outputFolder: htmlReportPath, open: 'never' }],     // Built-in HTML reporter
    ['junit', { outputFile: path.join(outputPath, 'junit-report.xml') }], // JUnit XML
    ['list'],                                                     // Console list reporter
    ['./src/support/reporter/appinsights-reporter.ts']           // Our custom reporter
  ],

  // ... rest of configuration
});

2. Reporter Loading and Instantiation

When Playwright loads the custom reporter, it follows this process:

// appinsights-reporter.ts - Export function that returns reporter instance
export default function (): Reporter {
  return new AppInsightsReporter();
}

Key Points:

The file path is relative to the project root
The exported function is called once per test run
Multiple reporters can run simultaneously
Each reporter receives the same test events

3. Reporter Lifecycle Integration

The Playwright engine automatically calls our reporter methods:

Test Execution Flow:
┌─────────────────┐
│ Playwright CLI  │
└─────────────────┘
          │
          ▼
┌─────────────────┐
│ Load Reporters  │ ◄── Our custom reporter loaded here
└─────────────────┘
          │
          ▼
┌─────────────────┐
│   Run Tests     │
└─────────────────┘
          │
          ▼ (for each test)
┌─────────────────┐
│ onTestEnd()     │ ◄── Individual test telemetry sent
└─────────────────┘
          │
          ▼ (after all tests)
┌─────────────────┐
│ onEnd()         │ ◄── Artifacts uploaded, telemetry flushed
└─────────────────┘

Custom Reporter Implementation Breakdown

1. Core Reporter Structure

import { Reporter, TestCase, TestResult, FullResult } from '@playwright/test/reporter';

class AppInsightsReporter implements Reporter {
  private runTimestamp: string | null = null;

  // Called for each test completion
  onTestEnd(test: TestCase, result: TestResult) {
    // Process individual test results
  }

  // Called when all tests complete
  async onEnd(result: FullResult): Promise<void> {
    // Process final results and upload artifacts
  }
}

2. Individual Test Processing (`onTestEnd`)

This method handles real-time telemetry for each test:

onTestEnd(test: TestCase, result: TestResult) {
  // Generate unique timestamp for this test run
  if (!this.runTimestamp) {
    this.runTimestamp = new Date().toISOString().replace(/[:.]/g, '-');
  }

  log(`[Reporter] onTestEnd called for test: ${test.title} with status: ${result.status}`);

  // Handle retry logic - only report final attempts
  const retries = test.retries || 0;
  const isLastRetry = result.retry === retries;
  const testPassed = result.status === 'passed';

  const shouldSendTrace = testPassed || isLastRetry;
  if (!shouldSendTrace) {
    log(`[Reporter] Skipping report for intermediate retry: ${test.title} (retry ${result.retry})`);
    return;
  }

  // Send availability telemetry to Application Insights
  const duration = result.duration || 0;
  const success = result.status === 'passed';

  appInsightsClient.trackAvailability({
    name: test.title,
    success,
    duration,
    runLocation: 'Azure Function - Playwright Synthetic Monitoring',
    message: success ? 'Test passed' : `Test failed: ${result.error?.message}`,
    time: new Date(this.runTimestamp),
    id: test.id,
  });
}

Key Features:

Retry Handling: Only sends telemetry for the final retry attempt to avoid duplicate data
Availability Tracking: Uses Application Insights availability telemetry for uptime monitoring
Rich Context: Includes test duration, success status, and error messages
Unique Identification: Each test run gets a timestamp-based identifier

3. Final Results Processing (`onEnd`)

This method handles artifact storage and telemetry flushing:

async onEnd(result: FullResult): Promise<void> {
  const shouldUpload = result.status === 'failed';

  log(`[Reporter] onEnd called. Test status: ${result.status}. Should upload: ${shouldUpload}`);

  // Upload artifacts only on failure
  if (shouldUpload) {
    try {
      await this.uploadTestArtifacts();
    } catch (err) {
      logError('[Reporter] Failed to zip or upload report:', err);
    }
  }

  // Ensure all telemetry is sent
  try {
    await flushTelemetry(5000);
  } catch (err) {
    logError('[AppInsights] Error during flush:', err);
  }
}

Key Features:

Conditional Upload: Only uploads artifacts when tests fail
Error Handling: Graceful handling of upload failures
Telemetry Flushing: Ensures all data reaches Application Insights before function ends

Application Insights Integration Deep Dive

1. Client Configuration

import * as appInsights from 'applicationinsights';

const connectionString = process.env.APPLICATIONINSIGHTS_CONNECTION_STRING;

appInsights
  .setup(connectionString)
  .setSendLiveMetrics(true)                    // Real-time monitoring
  .setDistributedTracingMode(appInsights.DistributedTracingModes.AI_AND_W3C)
  .setAutoDependencyCorrelation(true)          // Correlate related requests
  .setAutoCollectRequests(true)                // HTTP requests
  .setAutoCollectPerformance(true, true)       // Performance counters
  .setAutoCollectExceptions(true)              // Unhandled exceptions
  .setAutoCollectDependencies(true)            // External dependencies
  .setAutoCollectConsole(true)                 // Console logs
  .setUseDiskRetryCaching(true)               // Retry failed sends
  .setInternalLogging(true, true)             // Debug information
  .start();

const appInsightsClient = appInsights.defaultClient;

2. Telemetry Types and Usage

Availability Telemetry

Perfect for synthetic monitoring as it tracks uptime and response times:

appInsightsClient.trackAvailability({
  name: 'Login Flow Test',              // Test identifier
  success: true,                        // Pass/fail status
  duration: 2543,                       // Response time in ms
  runLocation: 'Azure Function',        // Where test ran
  message: 'Test passed',              // Additional context
  time: new Date(),                    // When test ran
  id: 'unique-test-id'                 // Correlation ID
});

3. Telemetry Flushing

Critical for Azure Functions to ensure data is sent before function terminates:

export async function flushTelemetry(timeoutMs: number = 5000): Promise<void> {
  return new Promise((resolve, reject) => {
    const timeout = setTimeout(() => {
      reject(new Error(`Telemetry flush timeout after ${timeoutMs}ms`));
    }, timeoutMs);

    appInsightsClient.flush({
      callback: (response) => {
        clearTimeout(timeout);
        if (response) {
          console.log('[AppInsights] Telemetry flushed successfully');
          resolve();
        } else {
          reject(new Error('Failed to flush telemetry'));
        }
      }
    });
  });
}

Azure Blob Storage Integration Deep Dive

Azure Blob Storage serves as the repository for test artifacts, including HTML reports, screenshots, videos, and traces. This section explains in detail how the storage functions work and what they accomplish.

1. Storage Client Setup and Initialization

The blob storage client is initialized once and reused throughout the application lifecycle:

import { BlobServiceClient, ContainerClient } from '@azure/storage-blob';

const connectionString = process.env.AZURE_STORAGE_CONNECTION_STRING;
const containerName = process.env.BLOB_CONTAINER_NAME || 'test-artifacts';

let containerClient: ContainerClient | null = null;

if (connectionString) {
  const blobServiceClient = BlobServiceClient.fromConnectionString(connectionString);
  containerClient = blobServiceClient.getContainerClient(containerName);
} else {
  console.warn('[BlobStorage] Connection string not provided.');
}

export { containerClient };

What this does:

Connection Management: Establishes a connection to Azure Storage using the connection string
Container Reference: Gets a reference to the specific container where artifacts will be stored
Error Handling: Gracefully handles missing configuration without breaking the application
Singleton Pattern: Creates one client instance that's reused across the application

2. Artifact Upload Process - Detailed Breakdown

The upload process involves several critical steps, each with specific purposes:

Step 1: Report Compression Function

export async function zipReportFolder(folderPath: string, zipName: string): Promise<string> {
  const zipPath = path.join(os.tmpdir(), `${zipName}.zip`);

  await new Promise<void>((resolve, reject) => {
    const output = fs.createWriteStream(zipPath);
    const archive = archiver('zip', { zlib: { level: 9 } }); // Maximum compression

    output.on('close', () => {
      console.log(`[ZIP] Report successfully zipped at: ${zipPath}`);
      resolve();
    });

    archive.on('error', (err) => reject(err));

    archive.pipe(output);
    archive.directory(folderPath, false);  // Include all files in folder
    archive.finalize();
  });

  return zipPath;
}

Detailed Function Analysis:

File Path Generation:
- Creates a unique zip file path in the system's temporary directory
- Uses the provided zipName to ensure uniqueness across test runs
Stream Setup:
- createWriteStream: Creates a writable stream to the zip file
- archiver('zip', { zlib: { level: 9 } }): Creates a zip archiver with maximum compression
- Level 9 compression reduces file size but takes more CPU time
Content Processing:
- archive.directory(folderPath, false): Adds entire directory contents to zip
- false parameter means don't include the parent directory in the zip structure
- archive.finalize(): Completes the archiving process
What Gets Compressed:
- HTML test report files
- CSS stylesheets for report formatting
- JavaScript files for interactive features
- Test artifacts (screenshots, videos, traces)
- Any additional files in the report directory

Step 2: Blob Upload Function - Deep Analysis

export async function uploadFileToBlobStorage(filePath: string, blobName: string): Promise<void> {
  if (!containerClient) {
    console.warn('[BlobStorage] Container client not initialized.');
    return;
  }

  // Ensure container exists
  console.log('[BlobStorage] Checking/Creating container if not exists...');
  await containerClient.createIfNotExists();

  // Get blob client for specific file
  const blockBlobClient = containerClient.getBlockBlobClient(blobName);

  // Read and upload file
  console.log('[BlobStorage] Reading file...');
  const fileBuffer = fs.readFileSync(filePath);

  console.log('[BlobStorage] Uploading data...');
  await blockBlobClient.uploadData(fileBuffer, {
    blobHTTPHeaders: { 
      blobContentType: 'application/zip' 
    },
    metadata: {
      uploadedAt: new Date().toISOString(),
      source: 'playwright-synthetic-monitoring'
    }
  });

  console.log(`[BlobStorage] Upload completed: ${blobName}`);
}

Detailed Function Breakdown:

Pre-flight Validation:
- Checks if containerClient is initialized
- Gracefully exits if storage isn't configured (non-blocking failure)
- Prevents runtime errors in environments without blob storage
Container Management:
- createIfNotExists(): Ensures the storage container exists
- What this does: Creates the container if it doesn't exist, ignores if it does
- Why it's important: Handles first-time deployments and container deletion scenarios
- Permissions: Requires storage account contributor permissions
Blob Client Creation:
- getBlockBlobClient(blobName): Creates a client for the specific blob
- Block Blob: Optimized for streaming large files and parallel uploads
- Alternative Types: Page blobs (for VHDs), Append blobs (for logs)
- Naming: Uses the provided blobName which includes timestamp and test info
File Processing:
- fs.readFileSync(filePath): Reads the entire file into memory
- Synchronous Read: Simpler but blocks the thread
- Memory Considerations: Entire file loaded into RAM (consider streaming for large files)
- File Buffer: Binary representation ready for upload
Upload Operation:
- uploadData(fileBuffer, options): Uploads the file data to Azure
- HTTP Headers: Sets content type as 'application/zip' for proper handling
- Metadata: Adds custom metadata for tracking and organization
- Metadata Fields:
  - uploadedAt: Timestamp for retention policies and debugging
  - source: Identifies the origin system for multi-tenant scenarios
What Happens in Azure:
- File is stored in the specified container with the given name
- Metadata is indexed and searchable
- Content type enables proper browser handling when downloaded
- Azure automatically handles redundancy and durability

3. Storage Organization and Naming Strategy

The blob naming follows a structured pattern:

// Example blob names generated:
// report-2025-09-09T10-30-45-123Z.zip
// report-2025-09-09T14-15-22-456Z.zip

const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
const blobName = `report-${timestamp}.zip`;

Benefits of this naming strategy:

Chronological Sorting: Files sort naturally by creation time
Uniqueness: Timestamp ensures no naming conflicts
Readability: Easy to identify when tests ran
Automation Friendly: Parseable for cleanup scripts and analytics

3. Complete Upload Workflow

async onEnd(result: FullResult): Promise<void> {
  const shouldUpload = result.status === 'failed';

  if (shouldUpload) {
    try {
      // 1. Get report path
      const htmlReportPath = process.env.PLAYWRIGHT_HTML_REPORT || 
                           path.join(os.tmpdir(), 'playwright-html-report');

      // 2. Generate unique name with timestamp
      const timestamp = this.runTimestamp ?? new Date().toISOString().replace(/[:.]/g, '-');
      const zipName = `report-${timestamp}`;

      // 3. Compress report folder
      const zipPath = await zipReportFolder(htmlReportPath, zipName);
      log(`[Reporter] Zipped report at: ${zipPath}`);

      // 4. Upload to blob storage
      await uploadFileToBlobStorage(zipPath, `${zipName}.zip`);
      log('[Reporter] Upload complete.');

      // 5. Cleanup local file (optional)
      fs.unlinkSync(zipPath);

    } catch (err) {
      logError('[Reporter] Failed to zip or upload report:', err);
    }
  }
}

Conclusion

The custom Playwright reporter is the bridge between your test execution and monitoring infrastructure. By understanding how it processes results, sends telemetry to Application Insights, and uploads artifacts to Blob Storage, you can:

Monitor effectively with rich telemetry data
Debug efficiently with comprehensive artifacts
Scale reliably with optimized error handling
Maintain visibility into your synthetic monitoring health

This reporting mechanism transforms raw test results into actionable insights, enabling proactive monitoring and rapid issue resolution in your applications.

The key to success is balancing comprehensive reporting with performance, ensuring your monitoring doesn't become a bottleneck while providing the visibility needed to maintain application reliability.

Additional Resources

Exploring Azure Functions for Synthetic Monitoring with Playwright: A Complete Guide - Part 1

Nathan Araújo — Tue, 09 Sep 2025 16:04:46 +0000

Introduction

Synthetic monitoring is a proactive approach to monitoring web applications by simulating user interactions and measuring performance from the end-user perspective. This article demonstrates how to build a robust synthetic monitoring solution using Azure Functions and Playwright that can automatically test your web applications, report results to Azure Application Insights, and store test artifacts in Azure Blob Storage.

Why Synthetic Monitoring Matters

Traditional monitoring only tells you when something is already broken. Synthetic monitoring helps you:

Detect issues before users do by continuously running automated tests
Monitor critical user journeys like login, checkout, or key workflows
Validate deployments by ensuring core functionality works after releases

Architecture Overview

Our solution combines several Azure services to create a comprehensive monitoring system:

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Azure Timer   │    │  Azure Function │    │   Playwright    │
│   Trigger       │───▶│   (Runtime)     │───▶│   Test Runner   │
│  (Scheduled)    │    │                 │    │                 │
└─────────────────┘    └─────────────────┘    └─────────────────┘
                                                        │
                                                        ▼
                                              ┌─────────────────┐
                                              │  Test Results   │
                                              │   Processing    │
                                              └─────────────────┘
                                                        │
                                              ┌─────────┴─────────┐
                                              ▼                   ▼
                                     ┌─────────────────┐ ┌─────────────────┐
                                     │ Application     │ │   Azure Blob    │
                                     │ Insights        │ │   Storage       │
                                     │ (Telemetry)     │ │ (Artifacts)     │
                                     └─────────────────┘ └─────────────────┘

Prerequisites

Before starting, ensure you have:

Node.js 18+ installed
Azure CLI installed and configured
Azure Functions Core Tools v4
An Azure subscription with:
- Application Insights instance
- Storage Account
- Function App (Premium or Dedicated plan recommended)

Project Setup

1. Initialize the Project

# Create project directory
mkdir synthetic-monitoring
cd synthetic-monitoring

# Initialize Node.js project
npm init -y

# Install core dependencies
npm install @azure/functions @playwright/test playwright
npm install @azure/storage-blob applicationinsights archiver dotenv

# Install development dependencies
npm install -D typescript @types/node @types/archiver

2. Project Structure

Create the following directory structure:

synthetic-monitoring/
├── src/
│   ├── functions/
│   │   └── synthetic-monitor.ts      # Timer-triggered function
│   ├── support/
│   │   ├── azure/
│   │   │   ├── app-insights-client.ts
│   │   │   ├── app-insights.ts
│   │   │   ├── blob-storage-client.ts
│   │   │   └── blob-storage.ts
│   │   ├── logger/
│   │   │   └── logger.ts
│   │   ├── pages/                    # Page Object Models
│   │   ├── reporter/
│   │   │   ├── appinsights-reporter.ts
│   │   │   └── utils.ts
│   │   └── services/                 # Authentication services
│   ├── tests/
│   │   └── marketplace-integration.spec.ts
│   └── index.ts                      # Main entry point
├── host.json                         # Azure Functions configuration
├── local.settings.json              # Local environment variables
├── package.json
├── playwright.config.ts             # Playwright configuration
└── tsconfig.json                    # TypeScript configuration

Core Implementation

1. Azure Function Timer Trigger

Create the main timer-triggered function (src/functions/synthetic-monitor.ts):

import { app, InvocationContext, Timer } from '@azure/functions';
import { runPlaywrightTests } from '../index';

app.timer('syntheticMonitor', {
  schedule: process.env.SYNTHETIC_MONITOR_SCHEDULE || '|| '0 0 * * * *',
  handler: async (myTimer: Timer, context: InvocationContext) => {
    try {
      context.log("🚀 Executing synthetic monitoring tests...");
      await runPlaywrightTests(context);
      context.log("✅ Synthetic monitoring tests completed successfully!");
    } catch (error) {
      context.log("❌ Error in synthetic monitoring tests:", error);
      throw error; // Ensure the function fails for monitoring purposes
    } finally {
      context.log("🔄 Synthetic monitoring cycle completed.");
    }
  }
});

2. Main Test Runner

Implement the core test execution logic (src/index.ts):

import { spawn } from 'child_process';
import { setLoggerContext } from './support/logger/logger';
import { InvocationContext } from '@azure/functions';
import path from 'path';

// Import the function to register it
import './functions/synthetic-monitor';

export async function runPlaywrightTests(context: InvocationContext): Promise<void> {
  setLoggerContext(context);

  return new Promise((resolve, reject) => {
    context.log('▶️ Running Playwright tests via CLI...');

    // Use full path to playwright binary
    const playwrightPath = path.join(process.cwd(), 'node_modules', '.bin', 'playwright.cmd');
    const command = \`"\${playwrightPath}" test\`;

    context.log(\`📍 Using command: \${command}\`);
    const child = spawn(command, { 
      shell: true,
      env: {
        ...process.env,
        // Add any environment variables your tests need
        NODE_ENV: 'production'
      }
    });

    child.stdout.on('data', (data) => {
      context.log(data.toString());
    });

    child.stderr.on('data', (data) => {
      context.log(data.toString());
    });

    child.on('error', (error) => {
      context.log(\`❌ Playwright tests failed: \${error.message}\`);
      reject(error);
    });

    child.on('close', (code) => {
      if (code === 0) {
        context.log('✅ Playwright tests completed.');
        resolve();
      } else {
        context.log(\`❌ Playwright tests exited with code \${code}\`);
        reject(new Error(\`Playwright tests failed with code \${code}\`));
      }
    });
  });
}

3. Playwright Configuration

Configure Playwright for Azure Functions (playwright.config.ts):

import { defineConfig, devices } from '@playwright/test';
import fs from 'fs';
import os from 'os';
import path from 'path';

const tmpDir = os.tmpdir();
const outputPath = process.env.PLAYWRIGHT_OUTPUT_DIR || path.join(tmpDir, 'playwright-artifacts');
const htmlReportPath = process.env.PLAYWRIGHT_HTML_REPORT || path.join(tmpDir, 'playwright-html-report');

// Ensure directories exist
if (!fs.existsSync(outputPath)) {
  fs.mkdirSync(outputPath, { recursive: true });
}
if (!fs.existsSync(htmlReportPath)) {
  fs.mkdirSync(htmlReportPath, { recursive: true });
}

export default defineConfig({
  testDir: './src/tests',
  timeout: 180 * 1000,  // 3 minutes per test
  retries: 1,           // Retry failed tests once
  fullyParallel: true,
  workers: 3,           // Adjust based on your Azure Function plan
  outputDir: outputPath,

  use: {
    headless: true,     // Always run headless in Azure Functions
    viewport: { width: 1280, height: 720 },
    baseURL: process.env.baseUrl,
    screenshot: 'only-on-failure',
    trace: 'retain-on-failure',
    actionTimeout: 30000,
  },

  reporter: [
    ['html', { outputFolder: htmlReportPath, open: 'never' }],
    ['junit', { outputFile: path.join(outputPath, 'junit-report.xml') }],
    ['list'], 
    ['./src/support/reporter/appinsights-reporter.ts'] // Custom reporter
  ],

  projects: [
    {
      name: 'chromium',
      use: { 
        ...devices['Desktop Chrome']
      },
    }
  ],
});

4. Azure Functions Configuration

Configure the function app (host.json):

{
  "version": "2.0",
  "logging": {
    "applicationInsights": {
      "samplingSettings": {
        "isEnabled": true,
        "excludedTypes": "Request"
      }
    },
    "logLevel": {
      "default": "Information"
    }
  },
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[4.*, 5.0.0)"
  }
}

Sample Test Implementation

Create a comprehensive test (src/tests/do-somethinng.spec.ts):

import { test, expect, BrowserContext, Page } from '@playwright/test';

const aadUsername: string = process.env['username'] || '';
const aadPassword: string = process.env['password'] || '';
const baseURL: string = process.env['baseUrl'] || '';

let context: BrowserContext;
let page: Page;

test.beforeEach(async ({ browser: bro }) => {
    context = await bro.newContext();
    page = await context.newPage();
    // Init all pages here

    await page.goto(baseURL);

}); 

test('User should be able to do somethinng', { tag: ['@some-tag', ] }, async () => {
    // All the test implemention here

    //Arrange
    //Act
    //Assert
});

test.afterEach(async () => {
  await page.close();
  await context.close();
});

Deployment and Configuration

1. Environment Variables

Set up your environment variables in Azure Function App settings:

{
  "AzureWebJobsStorage": "DefaultEndpointsProtocol=https;AccountName=<storage>;AccountKey=<key>",
  "FUNCTIONS_WORKER_RUNTIME": "node",
  "FUNCTIONS_EXTENSION_VERSION": "~4",

  //ScheduleTime
   "SYNTHETIC_MONITOR_SCHEDULE": "0 */10 * * * *",

  // Test Configuration
  "username": "<test-user-email>",
  "password": "<test-user-password>",
  "baseUrl": "<base-url>",

  // Azure Services
  "APPLICATIONINSIGHTS_CONNECTION_STRING": "<app-insights-connection>",
  "APPINSIGHTS_INSTRUMENTATIONKEY": "<app-insights-key>",
  "AZURE_BLOB_CONNECTION_STRING": "<blob-storage-connection>",
  "AZURE_BLOB_CONTAINER_NAME": "<container-name>",
}

2. Package.json Scripts

Configure build and deployment scripts:

{
  "scripts": {
    "build": "tsc",
    "start": "func start --typescript",
    "test:local": "npx playwright test",
    "install:browsers": "npx playwright install"
  }
}

Conclusion

Implementing synthetic monitoring with Azure Functions and Playwright provides a powerful, scalable solution for proactive application monitoring. This approach offers:

Automated testing on a schedule
Comprehensive reporting via Application Insights
Artifact storage for debugging failures
Cost-effective scaling with Azure Functions
Real user simulation with Playwright

The key to success is choosing the right Azure Function plan, optimizing Playwright configuration for cloud execution, and implementing robust error handling and monitoring.

With this foundation, you can expand your monitoring to cover critical user journeys, API endpoints, performance metrics, and multi-region testing to ensure your applications deliver optimal user experiences around the clock.

Additional Resources

This implementation is based on a real-world production synthetic monitoring system that monitors applications and critical user workflows.

Filtering vs. Locating Elements by Text in Playwright: What’s the Difference?

Nathan Araújo — Wed, 02 Apr 2025 19:56:33 +0000

When writing automated tests with Playwright, selecting elements efficiently is crucial to ensure reliable and maintainable test scripts. A common scenario is selecting an element that contains specific text inside a table. Playwright provides two approaches for this:

.filter({ hasText: 'TEXT' }) - filters elements that contain the specified text.
.locator('text=TEXT') - finds elements that match the exact text.

Both methods can be useful, but understanding their differences will help you write more robust selectors. Let’s dive into a practical example.

Example Scenario: Selecting a Button in a Table Row

Imagine you have the following table:

<table>
  <tr>
    <td>Processing</td>
    <td><button>Retry</button></td>
  </tr>
  <tr>
    <td>COMPLETED</td>
    <td><button>View</button></td>
  </tr>
  <tr>
    <td>Pending</td>
    <td><button>Retry</button></td>
  </tr>
</table>

You want to select the View button that belongs to the row where the status is COMPLETED.

Using .filter({ hasText })

await expect(
        page.locator('tr')
            .filter({hasText:'COMPLETED'})
            .locator('button'))
      .toHaveText('View')

✅ Why is this approach better?

It first selects all <tr>elements in the table.
Then, it filters the rows that contain the text COMPLETED.
Finally, it finds the <button> inside the filtered row

Using .locator('text=')

await expect(
         page.locator('tr')
            .locator('text=COMPLETED')
            .locator('button'))
      .toHaveText('View');

❌ What’s the issue here?

.locator('text=COMPLETED') tries to find any element containing COMPLETED, even if it’s deep inside another nested element.
It may return unexpected results if there are multiple matching elements inside different rows.
The search is not restricted to the row (tr) but instead looks for any match in the selected context.

🔥 Key Takeaways

Approach	Behavior	Best Use Case
`.filter({ hasText: 'TEXT' })`	Filters elements that contain the specified text	Selecting elements within a parent scope (e.g., table rows)
`.locator('text=TEXT')`	Locates any element that exactly matches the text	Quick selection when structure is simple

✅ When to Use .filter({ hasText })?

When dealing with structured elements (tables, lists, div containers, etc.).
When the text is part of a larger element (like a row containing multiple elements).
When you need precision and don’t want to match unintended elements.

⚠️ When .locator('text=') Can Cause Issues?

If the text appears multiple times on the page, you may get unexpected matches.
If the element you’re targeting is deeply nested, Playwright may not find it reliably.
It doesn’t work well when the text is inside an element with other dynamic content.

🚀 So

If you’re working with structured elements like tables or lists, prefer using .filter({ hasText }). It ensures you only interact with elements that are within the expected scope.

Use .locator('text=') carefully, as it may return unexpected elements depending on the page structure.

Follow me on Instagran and Linkedin

See more in TheCollege

[Cypress] Precisa fazer uma instrução condicional baseado se um elemento existe?

Nathan Araújo — Tue, 21 May 2024 16:53:16 +0000

Tentamos ao máximo evitar que nossos testes precisem de instruções de condições, por quê? Porque se temos condicionais significa que em algum aspecto temos comportamentos que estão acontecendo que não temos o controle e isso pode ser ruim para a automação que necessita de estabilidade.

Imagina você trabalhar na sua automação com alguns dados e caso você não tenha o controle da manipulação desses dados, se esperar por um dado específico mas ele não aparecer, pois você não produziu a sua massa de dados para aquele teste, assim se garantiu que o que existia na base seria ideal, mas se provou que não é.

Eu coloco esse exemplo pois, da mesma maneira pode acontecer com alguns elementos que queremos que eles estejam presente na página, mas acabam não estando porque o estado atual da massa de dados é outro.

Por exemplo, você tem um teste que precisa verificar se um ususário é adicionado a uma lista, todos os usuários adicionados trazem seu avatar. Se você não está manipulando os dados dessa lista de usuários, você pode começar seu teste com alguns usuários ou até mesmo nenhum e você precisa saber a quantidade exata de usuários para quando adicionar o novo usuário fazer a validação do total final.
Dessa forma, a única forma de você saber a quantidade de usuários iniciais seria verificar a quantidade de elementos da lista na página, verificando assim quantos avatares aparecem antes de adicionar, correto?

Lembrando, é claro que existem outros meios para se fazer isso, o exemplo que estou dando é para quando temos um contexto que não conseguimos manipular os dados por uma API ou diretamente no banco de dados antes dos testes e temos que solucionar através da UI.

Sendo assim, você precisa antes de adicionar, verificar a quantidade desses avatares (sabendo que temos o mesmo componente que representa os avatares com um locator genérico para os mesmos), mas aí que surge um problema, se você apenas se usar de cy.get(avatarLocator).its('length'), o comando irá falhar caso o **avatarLocator **não exista, e você não quer que falhe, pois sabe que isso pode acontecer, pois pode não ter nenhum usuário na lista.

Então, como realizar essa condicional para verificar se existe o elemento ou não? Pois se não existir você saberá que a lista está fazia e poderá continuar com as próximas instruções sem seu teste falhar.

Perceba que com esse código podemos criar condições para aqueles comportamentos que não conseguimos gerenciar através de alguma manipulação de dados e evitamos que o teste falhe quando não deveria. A tomada de decisão é baseado se um elemento existe ou não. Eu dei esse exemplo, mas com certeza se pode utilizar dessa mesma lógica para outros exemplos que tiver enfrentando, como aqueles cenários onde determinado botão só aparece caso um dado estado do usuário seja atendido. Já passou por isso?

E aí, gostou? Deu para destravar aí nos seus testes?
Tem outras ideias? Compartilha com a gente!

Quer saber mais sobre mim, acesso aqui: Nathan Araujo

[Cypress] Encontrando o valor que não está vísivel dentro do componente ao inspecionar

Nathan Araújo — Tue, 14 May 2024 17:07:54 +0000

Já pegou um cenário onde você precisa buscar o valor de um elemento, você está vendo que o valor está renderizado na página, mas quando executa o comando, por exemplo, em cypress:

cy.get(locator).should('have.text', text)

acontece um erro e diz que o elemento não tem esse valor, mesmo você vendo esse valor na sua cara? =/

Alguns componentes não podem estar trazendo esse valor, e se você inspecionar você pode encontrar o elemento mais ou menos assim:

<input id="campo-id" minlength="1" name="campo-id" required="" type="number" class="form-input ng-valid" placeholder="Informe o valor da compra">

No elemento em si não é mostrado o valor, mas ele é mostrado para o usuário, como então conseguir buscar esse valor desse input pra fazer sua validação.

Você pode usar o commando para buscar esse valor com:

cy.get(locator).invoke('val').then(valor => {
  // Fazer suas ações/validações com esse valor
})

cy.get(locator).invoke('val').should('contain', valor);

Agora esse valor estará disponivel pra você.

Esse ocultamento pode ser por alguns motivos, desde como o componente está sendo desenvolvido, no qual o CSS que está sendo usado está mascaradno esse valor ou questão de atualização assíncrona do estado ou autalização do DOM.

Tente entender com seu time o real motivo, mas nesse meio tempo, já agora não deve ser um problema pra você, visto que consegues recuperar esse valor.

E aí, já tinha visto isso acontecer? Tem outra solução?
Se não, já salva a dica para quando enfrentar esse comportamento.

Debugando elementos que desaparem rapidamente da página.

Nathan Araújo — Tue, 07 May 2024 16:58:53 +0000

Eu já sofri com isso no passado, ao tentar inspecionar um elemento em uma página da web, ele desaparece rapidamente antes que você possa examiná-lo completamente. Isso torna difícil depurar problemas ou entender a estrutura da página que a gente quer automatizar.

Você está lá querendo identificar os elementos de forma correta, ver os atributos e de repente você se depara com uma tooltip que está desaparecendo, no qual só aparece quando você coloca o mouse em cima, mas logo desaparece quando remove o mouse ou é aquela lista que quando tenta inspecionar um elemento, ela some automaticamente. E você fica pensando, mas e agora, como que vou inspecionar esse elemento que estou precisando?

Fazendo uma nota rápida aqui:
Visto os exemplos iniciais que comentei (Apenas para elucidar o problema que quero ajudar a resolver no seu dia-a-dia), alguém pode dizer, mas de repente esse teste aí não seja para você automatizar nessa camada de UI, deveria ser um unitário ou componente. Não quero entrar nesse discussão agora, porque uma coisa é que talvez de fato não deveria, outra é, beleza, não deveria, mas no momento eu preciso automatizar, e quando essa realidade bate a porta, não vou mostrar como pode ser feito? Seria no mínimo ridículo, não é? haha Mas tem muita gente falando tanto de melhores práticas e estratégias que descolam da realidade do cotidiano de algum QA por aí, jogam as "melhores práticas" e não dão alguma solução provisória enquanto o contexto do projeto não muda. Percebe?

Mas voltemos aqui para entender como podemos debugar esse bendito elemento que some. Felizmente, você pode usar uma técnica simples no console das Ferramentas de Desenvolvedor (DevTools) do seu navegador para superar esse problema. Aqui está como fazer isso:

Abra as Ferramentas de Desenvolvedor do seu navegador (normalmente acessíveis pressionando F12 ou clicando com o botão direito do mouse em uma página e selecionando "Inspecionar" ou "Inspeção").
Na barra de ferramentas das Ferramentas de Desenvolvedor (DevTools), navegue até a aba Console.
Digite o seguinte comando no console e pressione Enter:

setTimeout(function() {
  debugger;
}, 5000);

Isso define um temporizador para 5000 milissegundos (5 segundos). Depois desse intervalo, o comando debugger será executado.

Após executar o comando, execute a ação que faz com que o elemento que você deseja inspecionar apareça na página (por exemplo, passe o mouse sobre um elemento, clique em um botão, etc.).
Quando o temporizador expirar, a execução do código será interrompida e você entrará no modo de depuração do console.

Agora você pode inspecionar o elemento e sua estrutura, mesmo que ele tenha desaparecido rapidamente anteriormente.

Essa técnica pode ser uma ferramenta útil ao trabalhar com páginas da web dinâmicas ou com elementos que desaparecem rapidamente. Experimente ajustar o tempo limite e usar outras técnicas de depuração para obter uma compreensão mais profunda da página que está inspecionando.

E aí, tava sofrendo pra inspecionar seu elemento?
Essa é uma das formas. Você sabe outras?
Compartilha aí para que mais pessoas possam ser ajudadas!

[Cypress] Melhorando suas esperas para diminuir aqueles flaky tests.

Nathan Araújo — Thu, 02 May 2024 13:48:16 +0000

Todos nós já passamos por momentos que testes ficam falhando, rodamos em nossas máquinas locais e tudo parace funcionar como deveria, mas quando começamos executá-los em uma pipeline, alguns testes começam a quebrar e pensamos: por que meu pai? por que está quebrando? Affs!

Existem motivos para que seus testes estejam quebrando quando não deveriam, ou seja, as regras de negócio estão corretas, nada de estranho para estar na aplicação que está sendo testada. Costumamos chamar essas intermitências de Flaky Tests. Afinal, nesse primeiro momento, não sabemos porque estão falhando, mas se rodarmos novamente, "misteriosamente" eles passam, ai ficam nessa indecisão de passar e falhar.

Quero hoje te ajudar com um dos motivos que podem estar causando esses Flaky Tests, um deles acontece porque hora os elementos são carregados mais rápidos e outra hora não, e como não estamos controlando isso, acabam que podem passar ou não dependendo da estabilidade da internet ou até mesmo da performance da sua aplicação.

Isso pode ser tratado de duas maneiras, sem se utilizar da má prática de se ter Implicit Waits, ou seja, sabe quando forçamos nossos testes esperarem um tempo definido, como: cy.wait(5000)? Seria isso. Não importa o que aconteceça, a automação terá que esperar esse tempo, mas não queremos isso.

A primeira opção que podemos aplicar para tentar corrigir esse Flaky Test que pode estar sendo causado por um elemento que ainda não apareceu, é aumentar o timeout do elemento que estamos procurando. O cypress automaticamente já possui um Explicit Wait, ou seja, ele sempre fica esperando que o elemento esteja visível e que consigamos interagir com o mesmo, mas o timeout default é de 4 segundos e pode ser pouco dependendo do nosso cenário, então, caso precise aumentar o timeout, basta adicionar o parâmetro de timeout quando buscar o seu elmento que está caausando o Flaky Test, dessa forma:

Assim, você não precisará colocar um cy.wait() e fica melhor a solução.

Mas pode ser que esse timeout ainda não esteja sendo suficiente porque afinal os elementos na tela só vão ser renderizados quando determinada request terminar.
Se esse for o caso, temos uma segunda solução, se você perceber que uma request ainda está em processamento e seu script de automação está prosseguir, sem esperar a resposta, e que isto está causando seu Flaky Test, você pode esperar até que a request finalize, dessa forma:

Mas lembre de aumentar o responseTimeout na configuração do cypress se quiser de forma global:

Ou se preferir, podes especificar o timeout direto no seu comando:

cy.wait('@getModules', { responseTimeout: 60000 });

Com essas duas soluções você pode diminuir mais seus Flaky Tests, caso sejam por problemas de que elementos não estão visíveis, aplicando assim melhores esperas seja pelo próprio elemento ou pela espera da finalização de alguma request.

Faz sentido para você? Tem sofrido com Flaky Test?
Espero que esse post possa te ajudar pelo menos com alguns deles!
Lembre-se, o exemplo pode ser em Cypress, mas se você entender o princípio disso, perceberá que vale para outros frameworks de automação também! Guarde isso!

DEV Community: Nathan Araújo

Streamline Your Test Automation with Azure Test Track VS Code Extension

Why This Process Matters: Microsoft's Automation Status Design

The Hidden Challenge with Azure DevOps Automation Status

Microsoft's Intelligent Design Philosophy

The Real Challenge

Microsoft's Recommended Workflow

What This Extension Solves

The Problem Every QA Faces

Introducing Azure Test Track Extension

What It Does

How It Works

Step 1: Add ADO_IDs Comment

Step 2: Run the Association Command

Step 3: See the Magic in Azure DevOps!

See It In Action

Before: Manual Test Case Management

Azure DevOps Integration Results

1. Associated Automation Tab - Populated Automatically

2. Automation Status Changed to "Automated"

VS Code Integration

1. Visual Indicators in Your Code

Quick Start Guide

1. Installation

2. Setup Environment Variables Globally

3. Start Using

Pro Tips for QA Teams

1. Batch Processing

2. Visual Verification

3. Multiple Test Cases

Real Impact on QA Productivity

Quality Improvements

Enterprise Ready

Security & Compliance

Team Collaboration

Need Help?

🔗 Quick Links

[Playwright] Element is outside of the viewport" in Playwright — why it happens and how to fix it

The problem

Why this happens

Real example

So Playwright couldn’t click because it wasn’t physically actionable.

Possible solution

1. Use page.evaluate with element.click()

2. Manually set checked and dispatch events

Investigation

Why document.getElementById("AcceptedTermsOfUse").click() works off-screen, but Playwright’s locator.click() does not?

Why Playwright behaves differently

Exploring Azure Functions for Synthetic Monitoring with Playwright: A Complete Guide - Part 5

Additional Resources

Exploring Azure Functions for Synthetic Monitoring with Playwright: A Complete Guide - Part 4

Azure Functions Deployment

Phase 1: Create Azure Resources

1. Create Function App

2. Create Storage Account (for artifacts)

3. Get Connection Strings

Phase 2: Configure Function App

Phase 3: Setup Azure DevOps Pipeline

1. Create Build Pipeline

2. Create Release Pipeline

Phase 3: Deploy and Monitor

1. Deploy

2. Monitor

Quick Checklist

Additional Resources

Exploring Azure Functions for Synthetic Monitoring with Playwright: A Complete Guide - Part 3

Running Synthetic Monitoring with Azurite and Local Testing

Local Development Architecture

Prerequisites

Setting Up Azurite for Local Blob Storage

1. Azurite Installation and Configuration

Global Installation Method:

2. Azurite Configuration Options

Local Environment Configuration

1. Local Settings Configuration

2. Complete Local Development Workflow

Terminal 1 - Start Azurite:

Terminal 2 - Build and Start Azure Functions:

Terminal 3 - Run Tests Manually (Optional):

Conclusion

1. Use `page.evaluate` with `element.click()`

2. Manually set `checked` and dispatch events

Why `document.getElementById("AcceptedTermsOfUse").click()` works off-screen, but Playwright’s `locator.click()` does not?

2. Individual Test Processing (`onTestEnd`)

3. Final Results Processing (`onEnd`)