DEV Community: Shell QA

How to Handle O365 Shared Mailboxes in CI/CD Test Automation

Shell QA — Sun, 16 Aug 2026 19:14:06 +0000

Testing email workflows—like OTP verification, password resets, or automated notifications—is a core requirement for robust end-to-end (E2E) automation.

However, automating Office 365 (O365) Shared Mailboxes via UI logins or legacy IMAP basic authentication is a nightmare. UI logins trigger MFA, IMAP basic auth is deprecated across Microsoft 365, and browser automation for Outlook web is notoriously flaky.

The enterprise-grade solution? Use Microsoft Graph API with Client Credentials (App-Only) Flow directly inside your automation framework.

Here is a step-by-step guide on how to set this up for your test automation pipelines.

Step 1: Azure AD (Entra ID) Configuration

To access a shared mailbox programmatically without interactive user logins or MFA:

Register an Application in the Azure Portal (Entra ID).
Under API Permissions, add Microsoft Graph permissions:

Mail.ReadWrite (Application Permission)
Mail.Send (Application Permission)

Grant Admin Consent for the permissions.
Generate a Client Secret (or upload a Certificate) under Certificates & secrets.
Save your Tenant ID, Client ID, and Client Secret securely in your environment variables.

💡 Security Best Practice: Restrict the app registration's access so it can only target the specific shared mailbox rather than all tenant mailboxes using an ApplicationAccessPolicy in Exchange Online PowerShell.

Step 2: Fetching OAuth 2.0 Tokens Programmatically

Before querying the mailbox, request a bearer token using the Azure AD token endpoint:

POST https://login.microsoftonline.com/{TENANT_ID}/oauth2/v2.0/token

Request Body (application/x-www-form-urlencoded):

client_id: {CLIENT_ID}
scope: https://graph.microsoft.com/.default
client_secret: {CLIENT_SECRET}
grant_type: client_credentials

Step 3: Querying the Shared Mailbox

Once you have the Bearer Token, make direct REST API requests targeting the shared mailbox email address:

1. Fetch Latest Unread Email (e.g., for OTP or Link Extraction)

GET [https://graph.microsoft.com/v1.0/users/shared-mailbox@yourdomain.com/messages?$filter=isRead](https://graph.microsoft.com/v1.0/users/shared-mailbox@yourdomain.com/messages?$filter=isRead) eq false&$top=1&$select=subject,body,receivedDateTime
Authorization: Bearer {YOUR_ACCESS_TOKEN}

2. Extract Links or Passcodes

Parse the returned JSON payload using Regex or HTML parsers to extract verification links, tokens, or body text directly in code—bypassing UI interaction completely.

3. Send Email via Shared Mailbox

POST [https://graph.microsoft.com/v1.0/users/shared-mailbox@yourdomain.com/sendMail](https://graph.microsoft.com/v1.0/users/shared-mailbox@yourdomain.com/sendMail)
Authorization: Bearer {YOUR_ACCESS_TOKEN}
Content-Type: application/json

{
  "message": {
    "subject": "Automated Test Notification",
    "body": {
      "contentType": "Text",
      "content": "Test run completed successfully."
    },
    "toRecipients": [
      {
        "emailAddress": {
          "address": "recipient@yourdomain.com"
        }
      }
    ]
  }
}

Why This Strategy Wins for CI/CD Pipelines

Fast & Reliable: Direct HTTP API execution takes milliseconds compared to seconds of UI navigation.
No MFA Blockers: App Credentials bypass multi-factor authentication seamlessly in headless Jenkins/GitHub Actions runs.
Flake-Free Execution: Zero UI flakiness from changing Outlook web interfaces or loading delays.

How are you currently handling email validations in your automation test suites? Let me know in the comments!

A Beginner's Guide to Performance Testing with Apache JMeter

Shell QA — Sun, 16 Aug 2026 18:52:42 +0000

Performance testing is essential for ensuring your applications can handle expected user loads without bottlenecks or failures. Apache JMeter remains one of the most popular open-source tools for load, stress, and performance testing.

Here is a quick guide to getting your JMeter environment set up and executing your first load test.

1. Prerequisites

JMeter requires Java to execute. Ensure you have JDK 11 or higher installed on your system.

Verify your Java installation:

java -version

2. Download and Installation

Download the latest binary zip/tgz file from the Official Apache JMeter Site.
Extract the archive into your preferred local directory.
Launch JMeter from the bin directory:
- Windows: Double-click jmeter.bat
- macOS/Linux: Open terminal and run ./jmeter.sh

3. Install the Plugins Manager

The Plugins Manager simplifies adding listeners, graph generators, and custom samplers.

Download jmeter-plugins-manager.jar from JMeter Plugins.
Move the file into your JMeter lib/ext directory.
Restart JMeter. Access the Plugins Manager under Options > Plugins Manager.

4. Building Your First Test Plan

Set up a basic HTTP test using the GUI interface:

Thread Group: Right-click Test Plan > Add > Threads (Users) > Thread Group. Configure your target virtual users, ramp-up time, and loop count.
HTTP Request Defaults: Right-click Thread Group > Add > Config Element > HTTP Request Defaults. Set your target server domain/IP and port.
HTTP Sampler: Right-click Thread Group > Add > Sampler > HTTP Request. Define the API path and request method.
Listeners: Right-click Thread Group > Add > Listener > View Results Tree or Summary Report (use these GUI listeners primarily for test script validation).

5. Running Tests in Non-GUI Mode

Never run actual heavy load tests through the JMeter GUI as it consumes significant local system resources. Use CLI mode for accuracy:

jmeter -n -t /path/to/testplan.jmx -l /path/to/results.jtl -e -o /path/to/html-report-folder

-n: Non-GUI execution
-t: Path to your .jmx test script
-l: File path to output raw results (.jtl)
-e -o: Automatically generates a full interactive HTML report dashboard after execution

Happy load testing!

Setting Up Playwright & Cucumber UI Tests in Azure DevOps with LambdaTest

Shell QA — Sun, 16 Aug 2026 18:47:10 +0000

Here is a step-by-step guide to configuring your Playwright/Cucumber test suite to run on LambdaTest Cloud via Azure DevOps pipelines, returning test results directly to Azure.

1. Prerequisites

A GitHub repository containing your Playwright, Cucumber, and JavaScript automation code.
An active Azure DevOps account with a project created.
A LambdaTest account (you will need your username and access key).

2. Connect GitHub to Azure DevOps

In Azure DevOps, navigate to Pipelines > New Pipeline.
Select GitHub as the source and authenticate your account.
Choose your repository and target branch (e.g., main).

3. Create LambdaTest Credentials Variable Group

Go to Pipelines > Library in Azure DevOps.
Click + Variable group and name it LambdaTest-Credentials.
Add the following key-value pairs:
- LAMBDATEST_USERNAME = your_lambdatest_username
- LAMBDATEST_ACCESS_KEY = your_lambdatest_access_key (toggle "Keep this value secret")
Save the group.

4. Add/Update Your azure-pipelines.yml

Place this configuration file in your repository root directory:

trigger:
- main

pool:
  vmImage: 'windows-latest'

variables:
- group: LambdaTest-Credentials
- name: BASE_URL
  value: 'https://your-app-url.com'
- name: LT_BROWSER
  value: 'chrome'
- name: ENABLE_LAMBDATEST
  value: 'true'

stages:
- stage: Test
  jobs:
  - job: UITestsLambdaTest
    displayName: 'UI Tests (LambdaTest Cloud)'
    steps:
    - task: NodeTool@0
      inputs:
        versionSpec: '20.x'
      displayName: 'Install Node.js 20.x'

    - script: npm ci
      displayName: 'Install Dependencies'

    - script: npm run test:ui:smoke
      displayName: 'Run UI Smoke Tests on LambdaTest'
      env:
        ENABLE_LAMBDATEST: 'true'
        LT_USERNAME: $(LAMBDATEST_USERNAME)
        LT_ACCESS_KEY: $(LAMBDATEST_ACCESS_KEY)
        LT_BROWSER: $(LT_BROWSER)
        BASE_URL: $(BASE_URL)

    - task: PublishTestResults@2
      condition: always()
      inputs:
        testResultsFormat: 'JUnit'
        testResultsFiles: 'reports/junit-report.xml'
        testRunTitle: 'UI Tests - LambdaTest Cloud'

5. Update Your Test Code

Ensure your test runner reads LT_USERNAME, LT_ACCESS_KEY, and ENABLE_LAMBDATEST from environment variables. Use these parameters to connect your runner to LambdaTest's WebSocket endpoint for remote browser execution.

6. Execution and Verification

Commit and push your changes to GitHub.
The push triggers your Azure DevOps pipeline automatically.
View execution logs and published JUnit reports in Azure DevOps under the pipeline run, or check live and historical video logs on your LambdaTest dashboard

The Ultimate Code Review Checklist for Data Validation Frameworks

Shell QA — Sun, 16 Aug 2026 18:24:04 +0000

A comprehensive, production-ready checklist for reviewing data validation, ETL testing, and automated reconciliation codebases.

Code reviews for data engineering tools need more rigor than standard web apps. A subtle bug in a data validation framework can cause silent pipeline failures, false positive test passes, or accidental execution of unbounded SQL queries on production warehouses.

Whether you are building a custom data framework or maintaining automated ETL tests, use this generalized checklist during code reviews to keep your test suites secure, performant, and reliable.

1. Test Case Configuration (YAML / JSON)

TC ID Matching: Ensure the tc_id value matches the configuration filename exactly.
Schema Validity: Verify that type (e.g., count, data, recon, file) and source/target drivers are valid and supported.
Explicit Enablers: Confirm the enabled field is explicitly set (true or false) rather than omitted.
Relative File Paths: For file-based validation, ensure paths are relative to defined source/target data directories.
Non-Empty Queries: Confirm SQL sources and targets include non-empty query strings or valid template paths.
Unique Case IDs: Ensure test case identifiers are unique across the test suite directory.
Documented Rationale: If a test case has enabled: false or uses numeric tolerance thresholds (validation_tolerance), ensure a comment explains the business reason.
Dependency Order: Verify that basic structural checks (COUNT) run prior to deep comparisons (DATA / RECON).

2. SQL & Query Logic

Explicit Projections: No SELECT *. All columns must be explicitly listed to avoid schema drift breaks.
Alignment: Source and target queries must return compatible data types and matching column ordering.
Environment Isolation: Check that query strings contain zero hardcoded hostnames, schema names, or environment paths.
Secret Hygiene: Ensure queries contain no hardcoded credentials or connection strings.
Warehouse Pushdown: Confirm filtering and heavy aggregation occur at the database level (WHERE / GROUP BY) rather than pulling full tables into application memory.
Determinism: Queries must return deterministic ordering (e.g., explicit ORDER BY on primary keys) so differential comparisons yield consistent results.

3. Validator Core Logic

Standardized Return Schema: Validator routines must always return a consistent payload schema (e.g., status, summary, src_row_count, tgt_row_count, matched_rows).
Status Consistency: Status values should use normalized uppercase strings ("PASS" / "FAIL").
Null/NaN Handling: Verify that NaN and NULL comparisons explicitly account for missing data parity.
Memory Guards: Row capping (e.g., .head(1000)) or chunking should be enforced to prevent OOM errors on large datasets.
Safe Numbers: Ensure tolerance thresholds enforce non-negative values (e.g., max(0.0, float(tolerance))).
Error Preservation: No silent exception swallowing — all except blocks must either re-raise or log through the logger framework.

4. Execution & Pipeline Runners

Type Dispatcher Safety: Ensure the execution runner raises an explicit ValueError when encountering an unsupported validation type.
Type Coercion Guards: Data loaders (e.g., CSV readers) should default string types (dtype=str) where appropriate to prevent silent type conversions (e.g., dropping leading zeroes in zip codes).
Isolation: Ensure individual test execution is wrapped in try/except blocks so a single failing test case doesn't collapse the entire run.
Output Capture: Redirect standard output streams properly so module logs are correctly captured in final HTML or JSON reports.

5. Security & Data Safety

No Hardcoded Credentials: Check that API keys, passwords, or connection strings are strictly retrieved from environment variables or secret managers.
No Raw PII: Confirm that local test fixtures (data/src, data/tgt) contain synthetic data instead of production PII or financial records.
Injection Prevention: SQL execution calls must use parameterized inputs instead of raw string formatting (f-strings).
Path Traversal Protection: Relative file operations must be validated to prevent directory traversal breakouts.

6. Code Quality & Maintenance

Logging: Ensure all operational feedback uses structured logging frameworks instead of raw print() statements.
Type Annotations: Function signatures must include type hints and clear docstrings.
Ignore Management: Verify that dynamic run artifacts (.pyc, pycache, local HTML reports, local .log files) are properly tracked in .gitignore.

Sign-Off Checklist Template

When reviewing a PR, drop this checklist template into your code review comment:

PR Code Review Sign-Off
- Config & SQL Review Passed
- Validator & Runner Logic Passed
- Security & Secret Scanning Passed
- Test Coverage Verified

How do you handle data testing in your pipelines? Let me know in the comments!

Building an Enterprise Data Validation Framework: From Architecture to Version Control

Shell QA — Sun, 16 Aug 2026 18:06:41 +0000

Introduction

When managing large-scale data migrations, ETL pipelines, or multi-database reconciliations, manual verification quickly falls short. Without structured automation, data discrepancies—ranging from missing rows to subtle rounding errors—easily slip into production.

To tackle this, we built a layered, configuration-driven Data Validation Framework in Python. This framework enables engineering and QA teams to define test cases using clean YAML files, run multi-level checks, and auto-generate executive HTML reports.

Here is the complete blueprint and best practices guide covering all 10 core architectural modules.

1. Framework Architecture Overview

The framework follows a layered, configuration-driven architecture. Understanding the execution flow prevents common misconfigurations.

run_regression.py
├── config/execution.yaml (Group-level ON/OFF switches)
├── testcases/TC_XXX_TYPE.yaml (Individual test case definitions)
├── execution/validation_runner.py (Routes to correct validator)
│   ├── execution/df_loader.py (Loads DataFrames from source/target)
│   │   ├── execution/csv_loader.py (CSV path)
│   │   └── execution/sql_executor.py (SQL path)
│   └── validators/
│       ├── count_validator.py
│       ├── data_validator.py
│       ├── recon_validator.py
│       └── file_validator.py
├── utils/html_reporter.py (Generates summary HTML)
└── utils/logger.py (Rotating file logger -> logs/framework.log)

Key Design Principle: Control operates at two levels—group-level (execution.yaml) and test-case-level (enabled flag in YAML). Both must be set to true for a test case to execute.

2. Test Case Authoring

Naming Convention & Required Fields

Follow a strict pattern: TC_{NNN}_{TYPE}.yaml (e.g., TC_001_COUNT.yaml).

tc_id: TC_001_COUNT  # Must match the filename exactly
enabled: true        # Set false to skip without deleting
type: count          # count | data | recon | file
description: ">"
  Validate row count between Source DB and Target Staging table

source:
  type: sql
  path: sql/src/TC_001_COUNT.sql

target:
  type: sql
  path: sql/tgt/TC_001_COUNT.sql

ID Alignment: Always ensure tc_id matches the filename.
Human-Readable Descriptions: Populate clear descriptions that explain what business metric is being validated—this text directly feeds executive HTML summary reports.
Disabling vs. Deleting: Use enabled: false to skip tests temporarily. Never delete YAML files, as keeping them preserves history and audit trails.

3. Validation Type Selection

Choose the right strategy based on performance requirements:

Type	Scenario	When to Use
COUNT	Row Count Check	Quick sanity check. Always run first in any validation sequence.
DATA	Cell Comparison	Full cell-by-cell row matching for live database queries returning identical schemas.
RECON	Numeric Reconciliation	Column sum reconciliation with custom tolerances (e.g., max $0.01 rounding threshold).
FILE	Flat File Comparison	In-memory comparison tailored specifically for CSV or flat file extracts.

Always Lead with COUNT: A count failure signals pipeline or load failures immediately before you waste compute resources on heavy cell-level checks.
Tolerance Rules: Set tolerance explicitly in YAML. Never inflate tolerance to hide true data discrepancies without documented business justification.

4. SQL Query Management

External .sql Files: Store queries in sql/src/ and sql/tgt/ rather than inline YAML strings to maintain clean version control.
Determinism & Schema Drift: Include explicit ORDER BY clauses to ensure deterministic row ordering during DataFrame comparison. Avoid SELECT * in production tests; explicitly declare column names to prevent silent failures caused by schema drift.
Query Symmetry: Source and target queries must return matching column names and data types to prevent misleading shape mismatches.

5. Data Source & Connector Management

Modular Connectors: Encapsulate database connections inside dedicated modules under connectors/ using connection pooling. Never hardcode credentials—read them from environment files.
CSV File Hygiene: Store flat files under data/src/ and data/tgt/. Never commit real production data to version control—use anonymized or masked datasets.
Explicit Column Mapping: Use an explicit column_map block in YAML when column names differ between source and target rather than relying on position-based auto-alignment.

6. Execution Configuration

Group Switches (execution.yaml): Enable or disable entire validation suites (e.g., toggle all RECON tests off during initial staging loads).
Execution Scope: Tests run alphabetically by filename. Use clean numbering ranges or isolated directories for distinct projects.
Fail-Safe Processing: Wrap execution calls in try/except blocks so a single broken query or test case does not crash the entire regression suite.

7. Reporting & Logging

HTML Summary Reports: Automatically compiled into report/TC_000_SUMMARY.html after every execution cycle, embedding branding assets directly as base64 images.
Memory Capping: To optimize report rendering, mismatched row displays are capped (e.g., top 1,000 mismatches), with a "Download Full CSV" option embedded for deep root-cause analysis.
Log Rotation: Maintain a rotating file handler (e.g., 5 MB per log, 3 backups) at logs/framework.log set to INFO level to maintain complete audit trails.

8. File & Data Handling

NaN / Null Logic: Comparison modules enforce NaN-aware equality checks (NaN == NaN), treating matching empty/null positions across datasets as valid matches.
Numeric Coercion: Coerce string columns to numeric during RECON runs only when the operation is lossless, preventing silent data loss when reading flat files.
Sort Stability: Apply stable sorting algorithms (kind='stable') across all DataFrame columns prior to comparison for deterministic results.

9. Environment Management

Config Isolation: Isolate connection parameters by environment (config/env/qa.yaml, config/env/uat.yaml, config/env/prod.yaml).
Secrets Security: Inject sensitive credentials using environment variables or dedicated secret managers (Azure Key Vault, AWS Secrets Manager). Never commit raw passwords to repository control.

10. Framework Extension & Maintenance

Adding New Validators: Build custom modules under validators/ (e.g., schema_validator.py) adhering strictly to the framework's output contract (status, summary, src_to_tgt, tgt_to_src, matched_rows).
Connector Resiliency: Implement exponential backoff and retry logic in connectors to handle transient network blips gracefully.
Version Control Hygiene: Maintain a robust .gitignore excluding generated reports (report/.html), execution logs (logs/.log), temporary flat files (data/*/.csv), and pycache/.

Conclusion

A configuration-driven data validation framework provides the predictability and speed needed for modern data engineering pipelines. By decoupling test configuration from execution logic, teams can scale coverage effortlessly while maintaining high reliability.

How do you handle automated data validation in your ETL and pipeline workflows? Let's discuss in the comments below!

Building a Secure Enterprise AI Assistant: A Complete Architecture & Usage Guide

Shell QA — Sun, 16 Aug 2026 17:13:53 +0000

Integrating generative AI into internal workflows requires a strict balance between user accessibility and enterprise data privacy. Below is a blueprint for designing, deploying, and governing an internal AI assistant (Secure GPT) using managed LLMs.

1. Architecture & System Overview

An enterprise AI assistant acts as a secure bridge between internal teams and Large Language Models (LLMs).

Model Orchestration: Powered by managed endpoints (e.g., Azure OpenAI running models like GPT-4.1 Nano) to ensure consistent performance.
Data Boundary: Processing occurs entirely within isolated enterprise boundaries. Inputs are never retained, logged for third-party training, or exposed externally.
Context-Bound Execution: The assistant operates without live web access, relying strictly on curated training data cutoff points and user-provided session context to eliminate unauthorized external data leakage.
Multi-Modal Processing: Supports native parsing of structured documents and image inputs for real-time extraction.

2. Integration via API

For teams integrating the AI assistant into automated pipelines or internal tools:

// Example JSON request payload structure
{
  "model": "gpt-4.1-nano",
  "messages": [
    {
      "role": "system",
      "content": "You are an internal assistant. Follow data privacy guidelines."
    },
    {
      "role": "user",
      "content": "Summarize the key compliance points from the attached document."
    }
  ],
  "temperature": 0.2
}

Authentication: Access is managed through enterprise API gateways using scoped API keys.
Validation: Always validate outputs programmatically before passing generated responses to critical downstream business logic.

3. Prompt Engineering Best Practices

To help non-technical and technical users extract high-quality outputs, encourage these prompting patterns:

Specify Constraints: Replace broad requests with bounded requirements.
- Bad: "Tell me about our travel policies."
- Good: "Summarize the top 5 expense limits in our 2026 travel policy."
Provide Explicit Context: Frame the prompt with domain background (e.g., "Under GDPR compliance standards, how should we structure this data retention notice?").
Chain-of-Thought Decomposition: Break multi-step logic into distinct tasks within the prompt (e.g., "Step 1: Extract the core features. Step 2: Compare them against the baseline.").

4. Security & Governance Rules

Data Minimization: Avoid sending PII or sensitive system credentials unless explicitly isolated within secure pipeline boundaries.
Document Parsing: Encourage users to upload files directly into the context window rather than pasting raw text into chat inputs.
Human-in-the-Loop: Implement mandatory review policies for high-stakes operational outputs generated by the LLM.

Supercharging Test Automation with Custom AI Agents and Secure GPT

Shell QA — Sun, 16 Aug 2026 16:37:56 +0000

Introduction

As software applications grow in complexity, traditional test design and automation engineering often become bottlenecks. Between incomplete test cases, inconsistent documentation, and missing context, teams waste substantial cycles simply preparing test assets.

To tackle these challenges, we built an AI-driven test automation pipeline combining Custom AI Agents for script generation and Secure GPT for high-speed test design—all while keeping a Human-in-the-Loop for validation.

Phase 1: AI Custom Agents for Script Generation

Instead of using a single monolithic prompt, we broken down script generation into specialized, modular AI Agents that handle specific artifacts across the automation lifecycle.

The Agent Pipeline Architecture

Context Agent: Processes inputs like KT video recordings, application screenshots, and page sources to establish domain context.
Test Case Agent: Maps context to structured test cases.
Feature File Agent: Converts test cases into Gherkin feature files for BDD workflows.
Page Object Agent: Generates Page Object Model (POM) element locators and structure.
Step Definition Agent: Generates the underlying code logic for execution.
Telemetry Layer: Tracks token usage and execution status across agents, outputting real-time data to an executive reporting dashboard.

Key Challenges Solved

Insufficient context for AI adoption.
Single-line or incomplete test cases in legacy repositories.
Lack of test case prioritization and coverage gaps across application modules.
Inconsistent documentation.

Phase 2: Test Design Acceleration with Secure GPT

By leveraging a Secure GPT instance with a Human-in-the-Loop review process, we targeted initial test case generation for existing regression suites before expanding into active sprint stories.

Productivity Breakdown

Metric	AI Team Member (Secure GPT)	Traditional Automation Engineer
Average Productivity	~35 test cases / day	~7.5 test cases / day
Productivity Gain	4.6x Higher	Baseline
Time to Create 100 Test Cases	~3 Days	~13–20 Days

In initial rollouts across enterprise applications, accuracy rates consistently ranged between 40–60% for fully automated initial drafts, allowing test leads to focus on refining edge cases rather than building test suites from scratch.

What's Next?

Our next milestone expands this pipeline to in-sprint user stories. By feeding detailed user stories, Business Requirement Documents (BRDs), application screenshots, and acceptance criteria directly into Secure GPT, the team can auto-generate new test scenarios as soon as a story enters the sprint.

How is your team integrating generative AI into your testing workflows? Let’s discuss in the comments below!

Complete Guide to Building a Scalable Java-Selenium Automation Framework

Shell QA — Sun, 16 Aug 2026 16:24:55 +0000

QA Automation - Best Practices Guide
Framework: Hybrid Test Automation Framework
Stack: Java - Selenium 4 - TestNG - Maven - ExtentReports
Project: Enterprise QA Automation

Table of Contents

Project Structure
Naming Conventions
Page Object Model (UI Pages)
Business Components
Test Scripts
Data Management
Waits & Synchronization
Assertions & Reporting
Parallel Execution
Run Manager & Configuration
ExtentReports
CI/CD & Pipeline
Error Handling
Code Quality
Environment Configuration
1. Project Structure The framework follows a strict layered architecture. Never mix responsibilities across layers. src/ └── test/ ├── java/ │ ├── allocator/ + Entry point & parallel execution engine │ │ ├── Allocator.java │ │ └── ParallelRunner.java │ ├── uiPages/ + Page Object locators ONLY (no logic) │ ├── businessComponents/ + Reusable page actions & validations │ ├── commonComponents/ + Grouped reusable component flows │ └── testscripts/ + Test scripts (keywords/steps only) └── resources/ ├── Run Manager.xlsm + Test execution control └── GlobalSettings.properties

Rules:

uiPages: Only By locators (no logic, no assertions).
businessComponents: Only reusable actions; never call driver directly in test scripts.
testscripts: Only orchestrate business component calls (keyword-driven).
allocator: Do NOT modify unless changing threading/reporting strategy.
1. Naming Conventions | Layer | Class Naming | Method Naming | |---|---|---| | uiPages | page_name_uipages.java | static final By FieldName | | businessComponents | pagePageName.java | camelCase() verb-first | | testscripts | ScriptName.java | execute() or step methods | Examples: // Good - uiPages locator public static final By btn_Submit = By.xpath("//button[@id='btnSubmit']/span");

// Good - business component method
public void fillFormDetails(String primaryField, String secondaryField) throws InterruptedException { }

// Bad - logic inside uiPages
public void clickSubmit() { driver.findElement(btn_Submit).click(); } // Don't do this in uiPages!

Locator ID Preferences (Priority Order):

By.id() — most stable
By.name()
By.cssSelector()
By.xpath() — only when no better option exists; avoid absolute XPaths Preferred: By.id("btnSubmit") By.cssSelector("button[data-id='submit']")

Acceptable:
By.xpath("//button[@id='btnSubmit']/span")

Avoid - fragile absolute XPath:
By.xpath("/html/body/div[2]/form/button[1]")

Page Object Model (UI Pages) Each UI page maps to one class in uiPages. Keep locators static and final. // Good structure package uiPages;

import org.openqa.selenium.By;

public class form_details_page {

// Group locators with comments for readability
// === Header Section ===
public static final By txt_referenceNumber = By.xpath("//input[@id='referenceNumber']");
public static final By txt_submissionDate = By.id("submissionDate");

// === Dynamic Locators - use methods ===
public static By get_recordLink(String recordId) {
    return By.xpath("//a[text()='" + recordId + "']");
}

}

Rules:

One class per page/module.
Use static final for all fixed locators.
Use static methods for dynamic/parameterized locators.
Group related locators with inline comments.
Do not import Selenium WebDriver or WebElement in UI pages.
1. Business Components Business components contain the actual Selenium interactions and extend GeneralComponents (which extends ReusableLibrary). // Good business component method public void fillFormDetails(String userName, String effectiveDate) { sendKeys(form_details_page.txt_userName, userName, "User Name"); sendKeys(form_details_page.txt_effectiveDate, effectiveDate, "Effective Date"); report_updateTestLog("Fill Form Details", "User: " + userName + " | Date: " + effectiveDate, Status.PASS); }

// Bad - raw driver calls inside a business component
driver.findElement(By.xpath("//input[@id='userName']")).sendKeys(userName);

Rules:

Always use GeneralComponents helper methods (clickElement, sendKeys, selectDropDownByValue, etc.) — never call driver.findElement() directly.
Every significant action should update the report via report_updateTestLog.
Use wait (FluentWait and explicitWait) defined in GeneralComponents — never use Thread.sleep().
Use PauseScript() only as a last resort; prefer explicit waits.
Keep methods atomic — one action per method when possible. FluentWait Usage: // Correct - use inherited waits wait.until(ExpectedConditions.visibilityOfElementLocated(signin_home_page.welcomeText));

Test Scripts
Test scripts are the entry point for each test case. They call business component methods in sequence.
// Good test script structure
public class TC_001_CreateNewRecord extends ScriptHelper {

public void execute() {
// Step 1: Login
pdf_login_new = new pdf_login_helper();
login.login();

// Step 2: Navigate to New Record
pdf_home_new = new pdf_home_helper();
home.clickNewRecord();

// Step 3: Fill form details
pdf_formDetails = new pdf_formDetails_helper();
formDetails.fillFormDetails(
    dataTable.getData("TestData", "UserName"),
    dataTable.getData("TestData", "EffectiveDate")
);

}
}

Rules:

Test scripts must not contain XPaths, locators, or driver calls.
Each step should be a business component call.
Use dataTable.getData() for test data — never hardcode values.
Keep test scripts short and readable (under 100 lines where possible).
One test script = one test scenario.
1. Data Management Excel Run Manager:
Control test execution in Run Manager.xlsm — set Execute column to Yes/No.
Use TestConfig_optionsID to reference browser/environment config from the TestConfig_options sheet.
Set IterationMode to RUN_ALL_ITERATIONS, RUN_ONE_ITERATION_ONLY, or RUN_RANGE_OF_ITERATIONS. Test Data: // Read Data from Excel data sheet String email = dataTable.getData("TestData", "Email"); String country = dataTable.getCommonData("country", "DataValue");

// Never hardcode test data
String email = "user@example.com"; // Don't do this

Data Best Practices:
| Rule | Reason |
|---|---|
| Use CommonData sheet for shared values (email, URL, etc.) | Single source of truth |
| Use scenario-specific sheets for test-specific data | Separation of concerns |
| Do not store passwords in plain text in Excel | Security |
| Use JavaFaker for generating random test data where applicable | Data independence |
// Using JavaFaker for random data
Faker faker = new Faker();
String productName = faker.company().name();
String productRef = faker.number().digits(8);

Waits & Synchronization Wait Hierarchy (use in this order): | Wait Type | When to Use | Config Key | |---|---|---| | wait (FluentWait) | Standard UI elements | wait in properties | | longwait (FluentWait) | Slow-loading pages/modals | longwait in properties | | ExpectedConditions.visibilityOf | Checking visibility before interaction | — | | ExpectedConditions.elementToBeClickable | Before clicking dynamic elements | — | | PauseScript() | Last resort only — unavoidable timing gaps | — | Best practice wait pattern: try { wait.until(ExpectedConditions.visibilityOfElementLocated(locator)); clickElement(locator, "Button Name"); } catch (TimeoutException e) { report_updateTestLog("Step", "Element not visible after timeout", Status.FAIL); }

// For elements that may or may not appear
try {
wait.until(ExpectedConditions.visibilityOfElementLocated(optionalElement));
clickElement(optionalElement, "Optional Button");
} catch (TimeoutException | NoSuchElementException e) {
report_updateTestLog("Step", "Optional element not present - skipping", Status.PASS);
}

// Never use
Thread.sleep(5000);

Configuring Wait Timeouts:
Set in GlobalSettings.properties:
wait=30
longwait=60

Assertions & Reporting Status Levels: | Status | When to Use | |---|---| | Status.PASS | Assertion passed / action successful | | Status.FAIL | Critical assertion failed — stops test | | Status.WARNING | Non-critical mismatch (e.g., dynamic external values) | | Status.DONE | Informational step (no assertion) | Good assertion pattern: if (driver.findElement(welcomeText).isDisplayed()) { report_updateTestLog("Login Validation", "Homepage loaded successfully", Status.PASS); } else { report_updateTestLog("Login Validation", "Homepage did NOT load", Status.FAIL); }

// Use framework helper for value comparison
validateActualExpectedValue(actualValue, expectedValue, "Field Name");

// Use WARNING for values dependent on external systems (APIs, DBs)
validateActualExpectedValueForUnpredictableStringValue(actualValue, expectedValue, "Calculated Field");

Log Message Best Practices:
// Good - descriptive, includes actual value
report_updateTestLog("Field Validation",
"Actual: " + actualValue + " | Expected: " + expectedValue, Status.PASS);

// Bad - too vague
report_updateTestLog("Check", "OK", Status.PASS);

Parallel Execution The framework supports three threading models via Allocator.java: | Method | Description | When to Use | |---|---|---| | executeTestBatch() | Fixed ThreadPoolExecutor | Production stable runs | | executeTestBatch_Virtual() | Fixed ThreadPool (active) | Default - best for most scenarios | | executeTestBatch_AutoThreadPool() | WorkStealingPool | Experimental - perf testing | Thread Safety Rules:
- Never use static mutable state in business components or test scripts.
- referenceNumber in GeneralComponents is static — use with caution in parallel runs; prefer passing values as parameters.
- Each ParallelRunner instance is independent — ensure no shared file/resource access without synchronization. // Thread-safe - instance variable private String referenceNumber;

// Risk in parallel - static shared variable
public static String referenceNumber = null; // Avoid in parallel scenarios

Configuring Threads:
GlobalSettings.properties:
NumberOfThreads=3

Recommended: Set threads based on available machine CPUs (e.g., threads = CPUs * 2).
Caution: Too many threads on a single machine causes driver conflicts and flaky tests.
1. Run Manager & Configuration Run Manager Sheet Columns: | Column | Value | Notes | |---|---|---| | Execute | Yes / No | Controls if test runs | | TestScenario | Package/folder name | Maps to test script folder | | TestCase | Class name | Exact Java class name | | IterationMode | RUN_ALL_ITERATIONS | Controls data iteration | | StepsToExecute/EndIteration | Number | For ranged iterations | | TestConfigurationID | Config name | Maps to TestConfigurations sheet | TestConfigurations Sheet: | Column | Example Values | |---|---| | ExecutionMode | LOCAL, REMOTE, GRID | | Browser | EDGE, CHROME, FIREFOX | | Platform | WINDOWS, LINUX | | ExecutionTimeout | 120, leave blank for auto | GlobalSettings.properties Key Properties: # Execution RunConfiguration=Regression # Sheet name in Run Manager NumberOfThreads=2 DefaultBrowser=EDGE DefaultExecutionMode=LOCAL DefaultPlatform=WINDOWS Environment=SIT # SIT or UAT

URLs

uialuat_SIT=https://sit.app.example.com
uialuat_UAT=https://uat.app.example.com

Waits (seconds)

wait=30
longwait=60

Reporting

ProjectName=AutomationProject
GenerateHTMLReport=false

ExtentReports
Reports are auto-generated in target/Reports/Extent Result/ExtentReport.html.
Best Practices:
- Always call extentReportFlush() at the end — already handled in Allocator.driveBatchExecution().
- Ensure report_updateTestLog() is used — it auto-attaches screenshots on failure.
- Never create a new ExtentReports instance inside test scripts. Viewing Reports: target/ └── Reports/ ├── Extent Result/ │ └── ExtentReport.html • Open in browser └── HTML Reports/ └── index.html
CI/CD & Pipeline
Maven Profiles:

Run via Allocator (Hybrid Framework) - DEFAULT

mvn clean test -PRunAllocator

Run via TestNG suite

mvn clean test -PRunTestNGTests

Pass run configuration at runtime

mvn clean test -PRunAllocator -DRunConfiguration=SIT_Smoke

Pass environment at runtime

mvn clean test -PRunAllocator -DRunConfiguration=SIT_Smoke -DEnvironment=SIT

Azure Pipeline (azure-pipelines.yml):

Ensure NumberOfThreads is set to match pipeline agent specs.
Use pipeline variables for environment switching instead of hardcoding.
Archive target/Reports/ as a pipeline artifact for post-run analysis. # Good pipeline variable usage
- task: Maven@3 inputs: goals: 'clean test' options: '-PRunAllocator -DRunConfiguration=$(RunConfig) -DEnvironment=$(Env)'

Error Handling Exception Handling Pattern: // Catch specific exceptions and report try { wait.until(ExpectedConditions.visibilityOfElementLocated(locator)); clickElement(locator, "Submit Button"); } catch (TimeoutException e) { report_updateTestLog("Submit", "Element not visible - TimeoutException", Status.FAIL); } catch (NoSuchElementException e) { report_updateTestLog("Submit", "Element not found in DOM", Status.FAIL); }

// For optional elements (may or may not appear)
try {
clickElement(optionalPopup, "Optional Popup OK");
} catch (NoSuchElementException | TimeoutException e) {
// Silently skip - optional element
}

// Never swallow exceptions silently without a log
try {
// ...
} catch (Exception e) {
// Empty catch - bad practice
}

Stop Execution on Critical Failure:
The ParallelRunner checks frameworkParameters.getStopExecution() — use this mechanism when a prerequisite test fails and subsequent tests cannot proceed.

Code Quality General Rules: | Rule | Detail | |---|---| | DRY | Extract repeated actions into GeneralComponents or CommonMethods. | | Single Responsibility | Each method does one thing. | | Descriptive Names | fillFormDetails() not step1(). | | No Magic Numbers | Use named constants or data table values. | | Comment Why, Not What | Code is self-explanatory; comments explain business logic. | | Remove Dead Code | Don't leave commented-out blocks; use version control instead. | Preferred Patterns: // Extract repeated logic into CommonMethods public String getProcessedReferenceNumber(String refNumber) { return refNumber.substring(0, refNumber.length() - 1) + "B"; }

// Use constants instead of magic strings
private static final String PASS_STATUS = "PASS";
private static final String ENVIRONMENT_SIT = "SIT";

// Null/empty guard
if (testData != null && !testData.isEmpty()) {
getTestControlObject().getReport().reportEvent("TestControlData", testData, Status.PASS);
}

Code Review Checklist:

No hardcoded URLs, credentials, or test data.
All driver.findElement() calls go through GeneralComponents helpers.
All XPaths are relative (not absolute).
All exceptions are caught and logged to report.
Report steps have meaningful descriptions.
No Thread.sleep() in code.
Run Manager updated for new test cases.
UI page locators added to correct uiPages class.
1. Environment Configuration Switching Environments: Environment is controlled via GlobalSettings.properties or pipeline parameter: Environment=SIT or UAT Business components should always read URLs from properties: // Good - environment driven if (properties.getProperty("Environment").equals("SIT")) { driver.get(properties.getProperty("uialuat_SIT")); } else if (properties.getProperty("Environment").equals("UAT")) { driver.get(properties.getProperty("uialuat_UAT")); }

// Bad - hardcoded URL
driver.get("https://sit.app.example.com");

Supported Environments:
| Key | Description |
|---|---|
| SIT | System Integration Testing |
| UAT | User Acceptance Testing |
Quick Reference - Do's and Don'ts
| DO | DON'T |
|---|---|
| Use GeneralComponents helper methods | Call driver.findElement() directly in test scripts |
| Use FluentWait with ExpectedConditions | Use Thread.sleep() |
| Read all test data from Excel/properties | Hardcode values in test scripts |
| Report every step with report_updateTestLog() | Leave silent catch blocks |
| Keep locators in uiPages only | Put locators in business components |
| Use relative XPaths | Use absolute XPaths |
| Use Status.WARNING for dynamic external values | Fail tests for known external dependencies |
| Flush ExtentReport at end of run | Create multiple ExtentReport instances |
| Set Execute=No to skip tests | Delete rows from Run Manager |
| Use parameterized locator methods for dynamic elements | Concatenate XPath strings inline |
Support
Automation Team: QE Architect
automation.team@example.com
Generated for Hybrid Test Automation Framework — Enterprise QA Automation Project

Playwright JavaScript Framework Best Practices

Shell QA — Sun, 16 Aug 2026 16:13:21 +0000

Playwright JavaScript Framework — Best Practices
A comprehensive guide for writing reliable, maintainable, and scalable end-to-end tests using Playwright with JavaScript and Cucumber BDD.
Table of Contents

Project Structure & Organization
Page Object Model (POM)
Selectors & Locators
Assertions
Waiting Strategies
Test Isolation & State Management
Authentication & Login
Test Data Management
BDD / Cucumber Integration
Error Handling & Debugging
Retries & Flakiness
Parallelism & Performance
Configuration Management
Reporting & Observability
CI/CD Integration
Security & Secrets
Code Quality & Maintainability
Accessibility & Cross-Browser Testing
1. Project Structure & Organization DO
Keep a flat, predictable folder structure that mirrors the application's domain (e.g., admin/, user/, billing/, reports/).
Co-locate feature files, step definitions, and page objects by module/domain so related code is easy to find.
Use index.js barrel exports to avoid long relative import paths.
Store all environment-specific configuration in a single config.js at the root; never hardcode URLs or credentials inside test files. DON'T
Don't scatter page objects and step definitions randomly across the project.
Don't mix UI concerns with business logic in the same file. Recommended Layout features/ modules/ Admin_Group_Management.feature User_Task_Management.feature

step-definitions/
modules/
AdminSteps.js
UserSteps.js

page-objects/
modules/
basepage/
AdminPage.js
UserPage.js

utils/
logger.js
ExcelHelper.js

setup/
hooks.js
assertions.js
config.js

Page Object Model (POM)
DO
- Encapsulate all page interactions (clicks, fills, navigations) inside dedicated Page Object classes.
- Keep page objects thin — they should only expose methods, not assertions.
- Compose complex pages from smaller component objects (e.g., TableComponent, ModalComponent).
- Accept the Playwright page instance via the constructor; never create a new browser context inside a POM. // Good — page-objects/modules/AdminPage.js export class AdminPage { constructor(page) { this.page = page;
// Pre-define locators for reuse
this.groupNameInput = page.locator('[data-test="group-name"]');
this.saveButton = page.locator('[data-test="save-button"]');
this.successBanner = page.locator('[data-test="success-banner"]');
}

async navigateToGroupManagement() {
await this.page.click('[data-test="group-management"]');
}

async createGroup(groupData) {
await this.groupNameInput.fill(groupData.name);
await this.saveButton.click();
}
}

DON'T

Don't put expect() assertions inside page objects — keep them in step definitions or test files.
Don't duplicate selectors across multiple files; define them once in the page object.
1. Selectors & Locators Priority Order (most preferred → least preferred) | Priority | Strategy | Example | |---|---|---| | 1 | data-test / custom test data attributes | [data-test="save-button"] | | 2 | ARIA roles & labels | page.getByRole('button', { name: 'Save' }) | | 3 | Playwright built-in locators | page.getByLabel('Username') | | 4 | CSS class (stable, non-generated) | .modal-title | | 5 | XPath | //div[@class="header"] | DO
Use data-test or data-qa attributes — they are immune to styling and structural changes.
Use Playwright's semantic locators (getByRole, getByLabel, getByText, getByPlaceholder) for readability and resilience.
Define locators as class properties in page objects to avoid string duplication.
Use chaining to scope locators: page.locator('.modal').locator('[data-test="confirm"]'). // Semantic locator await page.getByRole('button', { name: 'Submit' }).click();

// custom data-test attribute
await page.locator('[data-test="group-name"]').fill('Automation Group');

DON'T

Don't use auto-generated class names (div.sc-abc123) or positional XPaths (/div[3]/span[1]).
Don't use page.$() (legacy Playwright API) — always use page.locator().
1. Assertions DO
Always use Playwright's built-in expect — it has automatic retry, built-in timeouts, and clear error messages.
Prefer web-first assertions that wait for the UI state to match: // Web-first assertions (auto-retry) await expect(page.locator('[data-test="success-banner"]')).toBeVisible(); await expect(page.locator('[data-test="user-count"]')).toHaveText('5'); await expect(page.locator('[data-test="save-button"]')).toBeEnabled();

// Use soft assertions when you want to collect multiple failures in one test run:
const softExpect = expect.configure({ soft: true });
await softExpect(heading).toHaveText('Dashboard');
await softExpect(logo).toBeVisible();
// All soft assertion failures are reported at the end

DON'T

Don't use page.isVisible() in if statements as a substitute for assertions.
Don't hard-code waitForTimeout before an assertion — let expect do the waiting.
1. Waiting Strategies DO
Rely on Playwright's auto-waiting — most click, fill, and expect operations auto-wait for elements to be actionable.
Use waitForSelector or waitForResponse only for specific async operations not covered by auto-waiting.
Wait for network responses when actions trigger API calls: // Wait for API response after action const [response] = await Promise.all([ page.waitForResponse(resp => resp.url().includes('/api/groups') && resp.status() === 200), page.locator('[data-test="save-button"]').click() ]);

// Use page.waitForLoadState('networkidle') only for pages with complex background requests.

DON'T

Never use arbitrary page.waitForTimeout(3000) — this is a top cause of slow, flaky tests.
Don't poll visibility in a loop; use expect(...).toBeVisible({ timeout: 10000 }) instead.
1. Test Isolation & State Management DO
Each Cucumber scenario must be fully independent — it should not rely on state left by a previous scenario.
Use Before / After hooks in setup/hooks.js to:
- Create a fresh browser context per scenario.
- Navigate to a known starting page.
- Clean up created test data after each scenario. // setup/hooks.js Before(async function () { this.context = await browser.newContext(); this.page = await this.context.newPage(); });

After(async function (scenario) {
if (scenario.result.status === 'FAILED') {
await this.page.screenshot({ path: reports/screenshots/${scenario.pickle.name}.png });
}
await this.context.close();
});

// Use tagged hooks to apply setup only to relevant scenarios:
Before({ tags: '@admin' }, async function () {
await loginAsAdmin(this.page);
});

DON'T

Don't share page instances or logged-in sessions across unrelated scenarios.
Don't depend on execution order — scenarios should be runnable in any order.
1. Authentication & Login DO
Reuse authenticated state using Playwright's storageState to avoid repeating login for every scenario: // Save auth state once await page.context().storageState({ path: 'setup/auth-state.json' });

// Reuse in playwright.config.js
use: {
storageState: 'setup/auth-state.json'
}

// Store credentials only in environment variables — never in code or feature files.
// Use a dedicated loginAsRole utility function to support multiple user roles cleanly.

// utils/auth.js
export async function loginAs(page, role) {
const creds = {
admin: { user: process.env.ADMIN_USER, pass: process.env.ADMIN_PASS },
user: { user: process.env.STANDARD_USER, pass: process.env.STANDARD_PASS },
approver: { user: process.env.APPROVER_USER, pass: process.env.APPROVER_PASS }
};

await page.goto(process.env.APP_BASE_URL);
await page.fill('[data-test="username"]', creds[role].user);
await page.fill('[data-test="password"]', creds[role].pass);
await page.click('[data-test="login-button"]');
await expect(page.locator('[data-test="dashboard"]')).toBeVisible();
}

Enterprise Single Sign-On (SSO)

Maintain a dedicated document (e.g., SSO_TESTING.md) for handling Identity Provider (IdP) login flows.
Mock or bypass SSO in lower environments whenever possible to speed up test execution.
1. Test Data Management DO
Keep test data separate from test logic — store in test-data/json/ or test-data/excel/.
Use unique data per run (e.g., timestamps, UUIDs) to prevent collisions when tests run in parallel. const groupName = AutoGroup_${Date.now()};

// Use factory functions to generate test data objects:
// utils/dataFactory.js
export function createGroupPayload(overrides = {}) {
return {
name: AutoGroup_${Date.now()},
description: 'Generated by automation',
type: 'standard',
...overrides
};
}

// Clean up all data created during a test in the After hook.

DON'T

Don't hardcode test data (names, IDs, dates) inside step definitions.
Don't leave orphaned test data in shared environments — it causes noise for manual testers.
1. BDD / Cucumber Integration Feature File Best Practices
Write scenarios from the user's perspective using Given / When / Then.
One scenario = one behavior. Don't write "super scenarios" that test 10 things at once.
Use Background for common pre-conditions, not complex setup logic.
Use tags consistently to allow selective execution: @smoke @regression @admin @group-management Feature: Admin Group Management

Background:
Given I am logged in as an admin

@create-group
Scenario: Admin creates a new group
When I navigate to Group Management
And I create a group with name "AutoGroup"
Then the group "AutoGroup" should appear in the list

Step Definition Best Practices

Keep steps atomic and reusable across scenarios.
Use World object (this) to share state between steps within a scenario — never use module-level globals.
Avoid logic-heavy step definitions; delegate to page objects.
Thin step, rich page object:
When('I create a group with name {string}', async function (name) {
await this.adminPage.createGroup(name);
});
Use Cucumber Data Tables and Doc Strings for structured input data.
1. Error Handling & Debugging DO
Enable screenshots on failure in After hooks (see Section 6).
Enable video recording for CI runs to replay failures:
// playwright.config.js
use: {
video: 'retain-on-failure',
screenshot: 'only-on-failure',
trace: 'retain-on-failure'
}
Use the Playwright Trace Viewer (npx playwright show-trace trace.zip) to inspect failing steps.
Use the built-in logger (utils/logger.js) instead of console.log for structured output.
Add meaningful step names so traces and reports are human-readable.
Debugging Locally

Run in headed mode with Playwright Inspector

PWDEBUG=1 npx playwright test

Run a single scenario by tag

npx test --tags="@create-group"

Slow down execution for visual debugging

npx playwright test --headed --slowmo=500

Retries & Flakiness Retry Policy
- Set retries: 1 (max 2) in playwright.config.js for CI — enough to handle transient network issues, not enough to hide real bugs.
- Never increase retries as a fix for a broken test — find and fix the root cause. // playwright.config.js retries: process.env.CI ? 1 : 0,

Flakiness Thresholds
| Level | Threshold | Action |
|---|---|---|
| Acceptable | < 1% over 7 days | Monitor |
| Warning | 1% – 5% | Investigate & RCA |
| Critical | > 5% | Block release, escalate |
Common Flakiness Causes & Fixes
| Cause | Fix |
|---|---|
| waitForTimeout | Replace with expect or waitForResponse |
| Fragile selectors | Switch to data-test / ARIA locators |
| Shared state | Isolate each scenario (see Section 6) |
| Race conditions | Use Promise.all + network wait |
| Dynamic content | Use toBeVisible({ timeout }) instead of hard-wait |

Parallelism & Performance DO
- Start with workers: 2 in CI and increase only after flakiness stabilizes below 1%.
- Use Playwright sharding to distribute tests across CI runners: npx playwright test --shard=1/3 npx playwright test --shard=2/3 npx playwright test --shard=3/3

Group slow scenarios (e.g., approval workflows) with @slow tags and run them separately.
Use storageState to avoid redundant logins (see Section 7). DON'T
Don't share a single browser context between parallel workers.
Don't run all scenarios in parallel before verifying they are properly isolated.
1. Configuration Management DO
Use a single config.js as the source of truth for all configuration values.
Override config values via environment variables — never change the config file between environments.
Use .env files for local development; inject secrets via CI environment variables in pipelines.
// config.js
export const config = {
baseUrl: process.env.APP_BASE_URL || 'https://qa-environment.example.com',
browser: process.env.BROWSER || 'chromium',
headless: process.env.HEADLESS !== 'false',
defaultTimeout: Number(process.env.DEFAULT_TIMEOUT) || 120000,
actionTimeout: Number(process.env.ACTION_TIMEOUT) || 30000,
};
See ENV_SETUP.md for full environment variable documentation.
DON'T
Don't commit .env files with real credentials to source control.
Don't hardcode environment URLs in test files — always reference config.js.
1. Reporting & Observability DO
Use Allure as the primary report for rich test history, attachments, and trend analysis.
Attach screenshots, videos, and traces to Allure on failure automatically via hooks.
Add Allure labels to enrich reports with metadata:
// in step definitions
this.allure.attachment('Response Body', JSON.stringify(responseBody, null, 2), 'application/json');
Generate reports as part of every CI pipeline run:
npm run generate:report # Generate Allure report
npm run open:report # Open in browser locally
Maintain a weekly flakiness dashboard from Allure history data and share with the team.
Report Locations
| Type | Location |
|---|---|
| Allure HTML | allure-report/ |
| Allure Results | allure-results/ |
| JSON Report | reports/cucumber_report.json |
| Screenshots | reports/screenshots/ |
| Videos | test-results/ |
1. CI/CD Integration DO
Run tests in headless mode in CI:
npm run test:ci
Fail the pipeline on any test failure — don't silently ignore failed tests.
Archive Allure results and test-results as CI artifacts for post-run analysis.
Run linting (eslint) and type checks before executing tests in the pipeline.
Use branch-specific tags to run only smoke tests on PRs and full regression on main:

Example CI step
- name: Run Smoke Tests (PR) run: npm test -- --tags "@smoke" if: github.event_name == 'pull_request'
- name: Run Full Suite (Main) run: npm test if: github.ref == 'refs/heads/main'

DON'T

Don't run tests against production environments from CI without explicit approval gates.
Don't skip report generation in CI — reports are essential for debugging failures.
1. Security & Secrets DO
Store all credentials in environment variables or a secrets manager (e.g., Vault, CI Secrets).
Reference secrets via process.env.VAR_NAME — never hardcode them.
Use .gitignore to exclude .env, auth-state.json, and any file that may contain tokens.
Rotate test user passwords regularly and update secrets in the CI store.
See SECRETS.md for detailed secrets management guidelines. DON'T
Don't log credentials or tokens to the console or report output.
Don't commit auth-state.json (contains session tokens) to source control.
Don't use personal accounts as test users — use dedicated service accounts.
1. Code Quality & Maintainability DO
Enable ESLint with a consistent style guide (e.g., Airbnb or StandardJS).
Use ES Modules (import/export) consistently — configure "type": "module" in package.json.
Use async/await everywhere — avoid mixing Promises and callbacks.
Keep functions small and single-purpose (\le 20 lines is a good target).
Write JSDoc comments for all page object methods and utilities.
Review test code in pull requests — test code is production code. /**
Creates a new group with the given data.
@param {Object} groupData
@param {string} groupData.name
@param {string} [groupData.description] */ async createGroup(groupData) { await this.groupNameInput.fill(groupData.name); await this.saveButton.click(); await expect(this.successBanner).toBeVisible(); }

DON'T

Don't leave commented-out code or console.log statements in committed code.
Don't duplicate step definitions — extract shared steps into a common-steps.js.
Don't use var — always use const or let.
1. Accessibility & Cross-Browser Testing Accessibility
Prefer ARIA roles and labels as primary selectors — this naturally validates accessibility.
Run @axe-core/playwright checks on key pages as part of accessibility regression: import { checkA11y } from 'axe-playwright';

await checkA11y(page, undefined, {
runOnly: ['wcag2a', 'wcag2aa']
});

Cross-Browser

Configure modern browsers (chromium, firefox, webkit) in playwright.config.js:
projects: [
{ name: 'chromium', use: { ...devices['Desktop Chrome'] } },
{ name: 'firefox', use: { ...devices['Desktop Firefox'] } },
{ name: 'webkit', use: { ...devices['Desktop Safari'] } }
]
Run cross-browser tests on a schedule (e.g., nightly), not on every PR, to keep PR pipelines fast.
Quick Reference Checklist
Use this checklist before merging new tests:
Tests are isolated — no shared state between scenarios
data-test or ARIA locators used — no fragile XPath/CSS
No waitForTimeout calls
Assertions use Playwright's expect (web-first)
Credentials are in environment variables, not hardcoded
Test data uses unique identifiers (timestamp/UUID)
Screenshots/video capture is configured on failure
Feature file has correct tags for selective execution
Page objects contain no assertions
Code reviewed and ESLint passes
References
Playwright Official Documentation
Playwright Best Practices
Cucumber.js Documentation
Allure Playwright Integration
ENV_SETUP.md
SELECTORS.md
FLAKINESS.md
SECRETS.md
SSO_TESTING.md
CI_HARNESS.md

Everything You Need for API Automation (A Complete Blueprint)

Shell QA — Sat, 15 Aug 2026 21:44:51 +0000

Setting up an API automation framework requires aligning business goals, developer specifications, infrastructure, and core testing strategies.

Here is a comprehensive requirement checklist and workflow to ensure complete coverage across every stage of your API automation setup.

1. Requirements from Client / Business Owner

Before writing code, define what needs to be tested:

Business requirements (BRD) & user stories / use cases
Expected API behavior & acceptance criteria (success & failure cases)
Priority APIs (critical vs optional pathing)
Performance expectations (SLA, response time)
API versioning policy (backward compatibility expectations)
Security & compliance requirements (data privacy, PII handling)

2. Technical Details from Developers

Understand how the APIs operate:

API Documentation: Swagger / OpenAPI specifications
Endpoints: Base URL + specific paths
HTTP Methods: GET, POST, PUT, DELETE, PATCH
Request Details: Headers, query params, request body (JSON/XML)
Response Details: Expected status codes (200, 201, 400, 401, 403, 404, 500) and response schema structures
Authentication: OAuth, JWT, API keys, or Basic Auth
Error Handling: Error codes & error messages
API Contracts: Consumer-driven contract definitions (e.g., using Pact)
Rate Limits & Throttling: Maximum request limits and wait strategies
Downstream Dependencies: Dependent APIs required for mock/stub planning

3. Infrastructure & Environment Setup

Coordinate with the Application Owner or Infra Team for execution requirements:

Environment URLs: Dev, QA, UAT, and Prod environments
Access Control: VPN access, API gateway setups, credentials
Test Data Strategy:
- Valid, invalid, edge case, and boundary value datasets
- Data seeding scripts for pre-test setup
- Data teardown/cleanup scripts for post-test cleanup
- Data isolation per environment
Database Access: Direct access for validating API output directly against DB records
Mocking/Stubbing: Availability of tools like WireMock or MSW for dependent APIs
Secrets Management: Environment variables (.env, secret managers) to avoid hardcoding sensitive tokens

4. Recommended Tooling Stack

Category	Recommended Tools
Manual Testing (First Step)	Postman (validate requests/responses, export collections to Newman)
Automation Tools	Playwright (JavaScript), RestAssured (Java), PyTest + Requests (Python), Karate
Performance Testing	k6, JMeter, Gatling
Security Testing	OWASP ZAP, Burp Suite
Contract Testing	Pact
Mocking / Stubbing	WireMock, MSW
CI/CD Integration	Jenkins, GitHub Actions, GitLab CI
Reporting	Allure Report, HTML Reporter, Extent Reports

5. Automation Framework Setup

Ensure your test automation codebase includes:

Data-driven, reusable design patterns
Reusable request utility and helper functions
Automatic authentication handling (token refreshes, session management)
Schema validation engines (JSON Schema / Ajv)
Comprehensive test suites covering:
- Positive test coverage (happy paths)
- Negative test coverage (missing fields, invalid data types, boundary values, malformed payloads, unauthorized access)
Multi-layer assertion strategy (status code + body structure + response headers + schema)
Pre-test data seeding and post-test data cleanup
Centralized logging and error reporting
Dynamic configuration management across environments
Code review guidelines prior to merging automation scripts

6. Pro Tips for API Automation

Manual First: Always validate endpoints manually in Postman before writing automated scripts.
Ensure API Stability: Wait until the endpoints are stable before investing time in script creation.
Keep Secrets Secret: Never hardcode credentials—manage access via environment variables or secret store managers.
Clean Up After Execution: Implement automated teardown routines to clean up test-created data.
Assert Schemas: Run schema validation on every response, not just status code checks.
Unified Frameworks: If using tools like Playwright, leverage the built-in API request context to combine API and UI tests within a single framework.

7. The Complete API Execution Workflow

Get API Details (Swagger/Docs)
       ↓
Manual Validation in Postman
       ↓
Schema & Contract Validation Setup
       ↓
Test Data Seeding
       ↓
Automate Scenarios (Positive + Negative + Edge Cases)
       ↓
Schema Assertions on Response
       ↓
Code Review & Approval
       ↓
CI/CD Pipeline Integration (Trigger on Build)
       ↓
Test Reporting (Allure / HTML Reports)
       ↓
Post-Test Data Cleanup (Teardown)
       ↓
Production Monitoring & Alerts

Must-Have Quick Verification Checklist

Swagger / API documentation received
Postman collections created & verified
Environment credentials and VPN access ready
Valid/Invalid test data seeded
Mock servers running for external dependencies
Test reporting integrated with CI/CD pipeline
Automated teardown configured

What tools are you currently using in your API automation stack? Let's discuss in the comments below!

Playwright + Cucumber Code Review Checklist: A Senior QA Guide to Reliable CI Test Suites

Shell QA — Sat, 15 Aug 2026 20:54:36 +0000

Building an enterprise test automation framework is one thing; keeping it reliable, maintainable, and fast in CI over time is another.

To prevent test rot, flakiness, and architectural drift, code reviews need to evaluate more than just standard syntax. They must enforce design boundaries, state isolation, and robust synchronization strategies.

Here is the complete Code Review Checklist we use for our Playwright + JavaScript + Cucumber BDD suites. Feel free to adopt or adapt it for your team's workflow!

Playwright JavaScript Framework — Code Review Checklist

Use this checklist during pull request reviews to keep tests reliable, maintainable, secure, and CI-friendly.

Scope

New Playwright test scenarios
Updated Cucumber feature files and step definitions
Changes to page objects, hooks, utilities, config, and test data
Reporting, retry, CI, and environment-related automation changes

Review Checklist

1. Test Intent and Coverage

Does the change clearly cover a real business workflow or acceptance criterion?
Is the scenario scoped to a single behavior instead of combining multiple unrelated validations?
Are positive, negative, and edge-case paths considered where appropriate?
Are feature names, scenario names, and tags meaningful and consistent?
Is there any duplication with an existing scenario that should be reused or refactored?

2. Feature File Quality (Cucumber)

Are Given, When, and Then steps written from the user's perspective?
Are scenarios concise, readable, and free of implementation details?
Are tags applied correctly for execution targeting (e.g., @smoke, @admin, @performer, @approver)?
Is Background used only for simple shared preconditions?
Are hardcoded credentials, URLs, IDs, or sensitive values avoided in feature files?

3. Step Definition Quality

Are step definitions thin and delegated to page objects or utilities?
Are duplicate or overly similar step definitions avoided?
Is shared scenario state stored on the Cucumber World object (this) instead of globals?
Are step definitions readable and reusable across scenarios?
Do step definitions avoid embedding large selectors, raw waits, or excessive business logic?

4. Page Object Design

Are UI interactions encapsulated in the correct page object under page-objects/?
Are selectors centralized in the page object instead of duplicated across steps?
Are page object methods named by user intent (e.g., createGroup, approveTask, searchQuery)?
Are page objects focused on actions and state access rather than test assertions?
If a page object became too large, should it be split into smaller components or helper classes?

5. Selector Strategy

Are selectors stable and intention-revealing?
Are data-qa, data-test, data-testid, or semantic Playwright locators preferred?
Are fragile selectors such as positional XPath, generated classes, or deep CSS chains avoided?
If a stable selector was not available, is the fallback rationale documented?
Are selector updates aligned with SELECTORS.md?

6. Assertions and Validation

Are assertions using Playwright's expect with web-first behavior?
Do assertions verify outcomes that matter to the user or workflow?
Are assertions placed in the appropriate layer rather than hidden inside unrelated helpers?
Are weak checks avoided, such as validating only that a page loaded without confirming expected data or state?
Where useful, do assertion messages or attachments help future debugging?

7. Waiting and Synchronization

Does the code rely on Playwright auto-waiting wherever possible?
Are explicit waits tied to real signals like element state, navigation, or API responses?
Are arbitrary sleeps such as waitForTimeout() avoided?
If network-dependent behavior exists, does the code wait on the relevant response or UI state change?
Could any race condition appear under CI speed or parallel execution?

8. Test Isolation and State Management

Can the scenario run independently and in any order?
Is test setup and teardown handled through hooks where appropriate?
Are browser context, page, storage state, and created records isolated per scenario?
Is cleanup handled for created users, groups, files, or seeded data when needed?
Does the change introduce hidden dependencies on previously executed tests?

9. Test Data Handling

Is test data externalized to test-data/, factories, or utilities where appropriate?
Are unique identifiers used to avoid collisions in shared environments?
Are hardcoded dates, names, or IDs avoided unless they are intentionally fixed test fixtures?
Is seeded data documented and compatible with the current environment?
Are data generation and cleanup steps deterministic and maintainable?

10. Configuration and Environment Safety

Are environment-specific values sourced from config.js or environment variables?
Are .env values kept out of committed code and documentation examples unless intentionally sanitized?
Does the change behave safely across local, QA, and CI environments?
Are timeouts, retries, browser settings, and base URLs configurable rather than hardcoded?
Is the documentation updated if the new change adds or alters configuration requirements?

11. Authentication and Secrets

Are credentials sourced only from environment variables or approved secret stores?
Are login flows reusable and abstracted where appropriate?
Are session files, tokens, or auth-state artifacts excluded from source control if sensitive?
Are sensitive values omitted from logs, screenshots, attachments, and console output?
Is the implementation aligned with SECRETS.md and ENV_SETUP.md?

12. Flakiness, Retries, and Reliability

Does the change reduce or increase flakiness risk?
Are retries treated as a safety net rather than the primary fix?
If a flaky area is touched, was the root cause addressed instead of masking the problem?
Is the scenario safe to run under current workers and retry settings?
Is the change consistent with FLAKINESS.md?

13. Logging, Debugging, and Failure Diagnostics

Will a future failure be easy to diagnose from logs, screenshots, traces, and reports?
Are screenshots, traces, or attachments captured at the right level when failures occur?
Does the code use structured logging utilities instead of noisy console.log statements?
Are debug artifacts meaningful and free of sensitive data?
If the reviewer had to debug this in CI, would the report provide enough context?

14. Code Quality and Maintainability

Is the code easy to read, with clear function and variable names?
Are functions small and focused on one responsibility?
Is duplication avoided through helpers, shared steps, or utility methods?
Is the module style consistent with the codebase conventions?
Are unused imports, dead code, commented-out blocks, and temporary debugging code removed?

15. Hooks, Utilities, and Shared Framework Code

If setup/hooks.js or shared utilities changed, have downstream impacts been considered?
Are hooks minimal, predictable, and safe for all tagged scenarios they affect?
Do shared utilities remain generic enough for reuse?
Could the change unintentionally affect unrelated modules like Admin, Performer, Approver?
If framework behavior changed, should related docs or runbooks be updated?

16. Parallelism and CI Compatibility

Will the change behave correctly when tests run in parallel or rerun on failure?
Are file paths, downloads, screenshots, and artifacts unique per test where needed?
Does the change assume a local-only environment or headed execution?
Is the scenario suitable for CI headless mode?
Are run tags and suite grouping still appropriate for smoke vs broader regression?

17. Reporting and Observability

Are Allure and other reporting outputs still generated correctly?
Are important artifacts attached for failures without creating excessive noise?
Does the change preserve report readability and trend usefulness?
If new metadata, labels, or attachments were added, are they helpful and consistent?
Are report-related updates aligned with current scripts and CI expectations?

18. Security, Compliance, and Governance

Does the change avoid exposing confidential business data in code, fixtures, or reports?
If AI-assisted code was introduced, does it receive human review per AI_GOVERNANCE.md?
Are compliance-sensitive workflows documented clearly enough for audit or review if needed?
Does the change follow team expectations captured in DoD.md?

Reviewer Quick Reference

Before hitting Approve, ask yourself:

Would this still be understandable to a new automation engineer in three months?
Would this remain reliable in CI under retry and parallel execution?
Would a failure provide enough evidence to debug quickly?
Did this change add unnecessary complexity to the framework?
Is the root cause solved, or is the PR only hiding symptoms?

How do you handle PR reviews for your automation suites? Let me know in the comments!

Automate Email Verification in Playwright using Microsoft Graph API & TypeScript

Shell QA — Sat, 15 Aug 2026 20:43:54 +0000

Automating end-to-end flows like user registration, password resets, or OTP verifications often hits a wall when dealing with emails. Automating UI login for webmail providers like Gmail or Outlook is flaky and frequently blocked by 2FA or CAPTCHAs.

Using Microsoft Graph API with an Azure App Registration (Client Credentials Flow) lets you fetch target user inbox messages programmatically in your Playwright tests without any UI interaction.

1. Prerequisites & Azure Setup

Before writing test code, configure your Azure Entra ID (formerly Azure AD):

Register an App: Go to Azure Portal > App registrations > New registration.
Set Permissions: Navigate to API permissions > Add a permission > Microsoft Graph > Application permissions.
Select Mail.Read (or Mail.ReadWrite if you plan to delete emails post-test).
Grant Admin Consent: Click Grant admin consent for [Your Org].
Create Credentials: Go to Certificates & secrets > New client secret. Store the secret securely.

Collect the following environment variables:

AZURE_TENANT_ID
AZURE_CLIENT_ID
AZURE_CLIENT_SECRET

2. Install Required Dependencies

Install the Azure Identity SDK and Microsoft Graph Client:

npm install @azure/identity @microsoft/microsoft-graph-client

3. Build the Graph Email Service

Create a dedicated helper service (graphService.ts) to manage authentication, inbox polling, and token extraction.

import { ClientSecretCredential } from '@azure/identity';
import { Client } from '@microsoft/microsoft-graph-client';
import { TokenCredentialAuthenticationProvider } from '@microsoft/microsoft-graph-client/authProviders/azureTokenCredentials';

export class GraphService {
  private graphClient: Client;

  constructor() {
    const tenantId = process.env.AZURE_TENANT_ID!;
    const clientId = process.env.AZURE_CLIENT_ID!;
    const clientSecret = process.env.AZURE_CLIENT_SECRET!;

    const credential = new ClientSecretCredential(tenantId, clientId, clientSecret);
    const authProvider = new TokenCredentialAuthenticationProvider(credential, {
      scopes: ['[https://graph.microsoft.com/.default](https://graph.microsoft.com/.default)'],
    });

    this.graphClient = Client.initWithMiddleware({ authProvider });
  }

  /**
   * Polls the inbox for an email matching specific subject criteria within a timeout.
   */
  async getLatestEmail(targetEmail: string, subjectContains: string, timeoutMs = 30000): Promise<string> {
    const startTime = Date.now();

    while (Date.now() - startTime < timeoutMs) {
      const response = await this.graphClient
        .api(/users/${targetEmail}/messages)
        .filter(contains(subject, '${subjectContains}'))
        .select('subject,body,receivedDateTime')
        .orderby('receivedDateTime desc')
        .top(1)
        .get();

      if (response.value && response.value.length > 0) {
        return response.value[0].body.content;
      }

      // Wait 3 seconds before polling again
      await new Promise((resolve) => setTimeout(resolve, 3000));
    }

    throw new Error(Email with subject containing "${subjectContains}" was not received within ${timeoutMs}ms.);
  }

  /**
   * Helper to extract links or OTP codes using regular expressions.
   */
  extractPattern(htmlContent: string, pattern: RegExp): string {
    const match = htmlContent.match(pattern);
    if (!match) {
      throw new Error(Pattern ${pattern} not found in the email content.);
    }
    return match[1] || match[0];
  }
}

4. Integrate with Playwright Tests

Use the service directly inside your Playwright test file (emailValidation.spec.ts):

import { test, expect } from '@playwright/test';
import { GraphService } from './graphService';

test.describe('Registration & Email Verification Flow', () => {
  const graphService = new GraphService();
  const testUserEmail = 'qa-automation@yourdomain.com';

  test('User registers and verifies OTP from email', async ({ page }) => {
    // 1. Trigger signup action in web app
    await page.goto('[https://example.com/register](https://example.com/register)');
    await page.fill('#email', testUserEmail);
    await page.click('#submit-btn');

    // 2. Fetch latest email and extract 6-digit OTP
    const emailBody = await graphService.getLatestEmail(testUserEmail, 'Your Verification Code');
    const otpCode = graphService.extractPattern(emailBody, /\b\d{6}\b/);

    // 3. Complete verification on page
    await page.fill('#otp-input', otpCode);
    await page.click('#verify-btn');

    // 4. Assert successful navigation
    await expect(page.locator('#dashboard')).toBeVisible();
  });
});

Key Takeaways

No Flaky UI: API-level retrieval eliminates UI dependency for email checking.
Polling Strategy: Always implement retry loops with timeouts to account for network latency in email delivery.
Security: Limit application permissions in Azure using Application Access Policies if you need to restrict access to specific automated QA mailboxes.

DEV Community: Shell QA

How to Handle O365 Shared Mailboxes in CI/CD Test Automation

Step 1: Azure AD (Entra ID) Configuration

Step 2: Fetching OAuth 2.0 Tokens Programmatically

Step 3: Querying the Shared Mailbox

1. Fetch Latest Unread Email (e.g., for OTP or Link Extraction)

2. Extract Links or Passcodes

3. Send Email via Shared Mailbox

Why This Strategy Wins for CI/CD Pipelines

A Beginner's Guide to Performance Testing with Apache JMeter

1. Prerequisites

2. Download and Installation

3. Install the Plugins Manager

4. Building Your First Test Plan

5. Running Tests in Non-GUI Mode

Setting Up Playwright & Cucumber UI Tests in Azure DevOps with LambdaTest

1. Prerequisites

2. Connect GitHub to Azure DevOps

3. Create LambdaTest Credentials Variable Group

4. Add/Update Your azure-pipelines.yml

5. Update Your Test Code

6. Execution and Verification

The Ultimate Code Review Checklist for Data Validation Frameworks

1. Test Case Configuration (YAML / JSON)

2. SQL & Query Logic

3. Validator Core Logic

4. Execution & Pipeline Runners

5. Security & Data Safety

6. Code Quality & Maintenance

Sign-Off Checklist Template

Building an Enterprise Data Validation Framework: From Architecture to Version Control

Introduction

1. Framework Architecture Overview

2. Test Case Authoring

3. Validation Type Selection

4. SQL Query Management

5. Data Source & Connector Management

6. Execution Configuration

7. Reporting & Logging

8. File & Data Handling

9. Environment Management

10. Framework Extension & Maintenance

Conclusion

Building a Secure Enterprise AI Assistant: A Complete Architecture & Usage Guide

1. Architecture & System Overview

2. Integration via API

3. Prompt Engineering Best Practices

4. Security & Governance Rules

Supercharging Test Automation with Custom AI Agents and Secure GPT

Introduction

Phase 1: AI Custom Agents for Script Generation

The Agent Pipeline Architecture

Key Challenges Solved

Phase 2: Test Design Acceleration with Secure GPT

Productivity Breakdown

What's Next?

Complete Guide to Building a Scalable Java-Selenium Automation Framework

URLs

Waits (seconds)

Reporting

Run via Allocator (Hybrid Framework) - DEFAULT

Run via TestNG suite

Pass run configuration at runtime

Pass environment at runtime

Playwright JavaScript Framework Best Practices

Run in headed mode with Playwright Inspector

Run a single scenario by tag

Slow down execution for visual debugging

Example CI step

Everything You Need for API Automation (A Complete Blueprint)

1. Requirements from Client / Business Owner

2. Technical Details from Developers

3. Infrastructure & Environment Setup

4. Recommended Tooling Stack

5. Automation Framework Setup

6. Pro Tips for API Automation

7. The Complete API Execution Workflow

Must-Have Quick Verification Checklist

Playwright + Cucumber Code Review Checklist: A Senior QA Guide to Reliable CI Test Suites

Playwright JavaScript Framework — Code Review Checklist