Smart Test Skipping: Building a Lightweight Playwright Dependency Analyzer

When an end-to-end test suite scales, so does the frustration from cascading failures. A single bug in a core component, like a LoginPage or HeaderPage, can cause dozens of dependent tests to fail, flooding your test report with noise. This makes it difficult to pinpoint the root cause.

A common solution is test dependency analysis: if a test for LoginPage fails, automatically skip all other tests that use LoginPage.

In this article, you'll build a simple, lightweight dependency analyzer that uses your Page Objects to provide smart "skip-if-failed" logic with minimal setup.

What We'll Build

1. An Analyzer: A single utility file that reads a test file and extracts its Page Object dependencies. It does this by looking at direct imports and new SomePage(...) instantiations.
2. A Fixture Integration: A simple override for the Playwright context fixture that:
   • Analyzes the test file before it runs.
   • Skips the test if any of its Page Object dependencies failed in a previous run.
   • Records the test's dependencies to a JSON file if it fails.

Let's get started.

---

Step 1: Create the Dependency Analyzer

First, we'll create the core logic in playwright/utils/testDependencyAnalyzer.ts. This file will contain all the functions needed to parse, record, and check for failures. We'll build it piece by piece.

Defining the Failure Record

To start, let's define the shape of the data we'll save to our JSON cache file. This type, FailureRecord, will store information about a failed test and its dependencies.

import * as fs from 'fs';
import * as fsp from 'fs/promises';
import * as path from 'path';

type FailureRecord = {
  testFile: string;
  testTitle: string;
  timestamp: string;
  errorFiles: string[]; // Files found in the error stack
  dependencies: string[]; // Page Objects this test uses
  testId: string;
  skipped?: boolean;
};

The Main Analyzer Function

This is the main function, analyzePageDependencies. It reads a test file, finds the project root, and then uses two methods to build a unique set of Page Object dependencies.

/**
 * Analyze a test file for Page Object dependencies.
 */
export function analyzePageDependencies(testFilePath: string): string[] {
  const content = fs.readFileSync(testFilePath, 'utf-8');
  const projectRoot = findProjectRoot(testFilePath); // Helper we'll define later
  const deps = new Set<string>();

  // 1. Find direct imports
  for (const p of extractDirectImports(content)) {
    const resolved = resolveImportPath(projectRoot, testFilePath, p);
    if (resolved && isPageObjectPath(resolved)) deps.add(resolved);
  }

  // 2. Find 'new PageObject()' instantiations
  for (const cls of extractPageClassInstantiations(content)) {
    const file = findPageObjectFileByClass(projectRoot, cls);
    if (file) deps.add(toRelative(projectRoot, file));
  }

  return Array.from(deps).sort();
}

Analysis Helpers: Parsing the Code

The main function relies on several helper functions to do the actual parsing.

1. Extracting Imports: We use regex to find all import statements and return the path.

/**
 * HELPER: Extract import sources.
 */
function extractDirectImports(content: string): string[] {
  const imports: string[] = [];
  const patterns = [
    /import\s+(?:{[^}]+}|[\w*]+)\s+from\s+['"]([^'\"]+)['"]/g,
    /import\s+['"]([^'\"]+)['"]/g,
    /import\s+\*\s+as\s+\w+\s+from\s+['"]([^'\"]+)['"]/g,
  ];
  for (const re of patterns) {
    let m;
    while ((m = re.exec(content)) !== null) imports.push(m[1]);
  }
  return imports;
}

2. Extracting Class Instantiations: We use another regex to find code like new SomePage(...). This is crucial for tests that might not import the page directly (e.g., if it comes from another helper).

/**
 * HELPER: Extract class names of `new SomePage(...)`.
 */
function extractPageClassInstantiations(content: string): string[] {
  const classes: string[] = [];
  // Look for 'new' followed by a class name ending in 'Page'
  const re = /new\s+(\w+Page)\s*\(/g;
  let m;
  while ((m = re.exec(content)) !== null) classes.push(m[1]);
  return Array.from(new Set(classes));
}

3. Identifying Page Objects: These helpers determine if a file path is a Page Object and find its file location by its class name. This is based on the convention that Page Objects live in playwright/pages or playwright/_redesign/pages.

/**
 * HELPER: Identify Page Object files by convention.
 */
function isPageObjectPath(relPath: string): boolean {
  return (
    relPath.includes('playwright/_redesign/pages') ||
    relPath.includes('playwright/pages') ||
    relPath.endsWith('.page.ts') ||
    relPath.endsWith('.page.tsx')
  );
}

/**
 * HELPER: Find a Page Object file by exported class name.
 */
function findPageObjectFileByClass(
  projectRoot: string,
  className: string,
): string | null {
  const dirs = [
    path.join(projectRoot, 'playwright/_redesign/pages'),
    path.join(projectRoot, 'playwright/pages'),
  ];
  for (const dir of dirs) {
    if (!fs.existsSync(dir)) continue;
    const files = getAllFiles(dir); // getAllFiles is a recursive file reader
    for (const f of files) {
      if (!/\.tsx?$/.test(f)) continue;
      const content = fs.readFileSync(f, 'utf-8');
      if (content.includes(`export class ${className}`)) return f;
    }
  }
  return null;
}

(Note: We'll include the other helpers like findProjectRoot, resolveImportPath, getAllFiles, and toRelative in the final code block. They are standard file/path utilities.)

Checking for Failures

This function, hasFailedPageDependencies, is used by our fixture before a test runs. It reads the failure cache and checks if any of the test's dependencies are on the list of previously failed files.

/**
 * Return true if any previously failed dependency matches current Page Object deps.
 */
export async function hasFailedPageDependencies(
  dependencies: string[],
): Promise<boolean> {
  if (!dependencies.length) return false;
  const failures = await readDependencyFailures(); // Helper to read JSON cache
  if (!failures.length) return false;

  return failures.some(
    (f) =>
      // Check if any error file matches a dependency
      f.errorFiles?.some((err) => matchesDependency(err, dependencies)) ||
      // Fallback: check error text
      errorTextContainsDependency(
        [(f as any)?.error?.message ?? '', (f as any)?.error?.stack ?? '']
          .join(' ')
          .toLowerCase(),
        dependencies,
      ),
  );
}

Recording Failures

This function, recordPageDependencyFailure, is used by our fixture after a test fails. It collects all file paths from the test's error stack and saves a new FailureRecord to our JSON cache.

/**
 * Record dependency failure for a test on failure/timeout.
 */
export async function recordPageDependencyFailure(
  testInfo: any,
  dependencies: string[],
): Promise<void> {
  if (!testInfo || !dependencies.length) return;

  // Find all file paths in the error stack
  const errorFiles = collectErrorFiles(testInfo);
  if (errorFiles.length === 0) return;

  const testId = testInfo.testId || 'unknown';
  const failures = await readDependencyFailures();

  // Don't record the same failure twice
  if (failures.find((x) => x?.testId === testId)) return;

  const record: FailureRecord = {
    testFile: testInfo.file || 'unknown',
    testTitle: testInfo.title || 'unknown',
    timestamp: new Date().toISOString(),
    errorFiles,
    dependencies,
    testId,
  };

  failures.push(record);
  await writeDependencyFailures(failures); // Helper to write to JSON cache

  if (Array.isArray(testInfo.annotations)) {
    testInfo.annotations.push({
      type: 'dependency-failure',
      description: "'Recorded Page Object dependency failure',"
    });
  }
}

Cache and Error Helpers

Finally, we need functions to read/write the cache file (tmp/dependency-failures.json) and to parse error stacks. The deletePageDependencyFailures function is exported so our globalSetup can clear the cache.

/**
 * Support util: delete the failures file.
 */
export async function deletePageDependencyFailures(): Promise<void> {
  const p = getFailuresPath();
  await fsp.unlink(p).catch(() => {}); // Ignore if file doesn't exist
}

/**
 * Persist/Load failures.
 */
function getFailuresPath(): string {
  const tmpDir = path.join(process.cwd(), 'tmp');
  return path.join(tmpDir, 'dependency-failures.json');
}

async function readDependencyFailures(): Promise<FailureRecord[]> {
  try {
    const p = getFailuresPath();
    const txt = await fsp.readFile(p, 'utf-8').catch(() => '[]');
    if (!txt.trim()) return [];
    const j = JSON.parse(txt);
    return Array.isArray(j) ? j : [];
  } catch {
    return [];
  }
}

async function writeDependencyFailures(
  failures: FailureRecord[],
): Promise<void> {
  const tmpDir = path.join(process.cwd(), 'tmp');
  await fsp.mkdir(tmpDir, { recursive: true });
  const p = getFailuresPath();
  await fsp.writeFile(p, JSON.stringify(failures, null, 2));
}

/**
 * Error helpers.
 */
function collectErrorFiles(testInfo: any): string[] {
  // ... (logic to parse testInfo.error, testInfo.result.error, etc.)
  // This function is complex and included in the full code block below
  // ...
  return []; // Placeholder for snippet
}

Complete Analyzer Code

Here is the complete code for playwright/utils/testDependencyAnalyzer.ts. This includes all the helper functions (like findProjectRoot, getAllFiles, collectErrorFiles, etc.) that were abbreviated in the snippets above.

/**
 * A dependency analyzer for Playwright tests based on Page Objects.
 * - Extracts dependencies from:
 * 1) Direct imports
 * 2) `new SomePage(...)` class instantiations resolved by scanning pages dirs
 */

import * as fs from 'fs';
import * as fsp from 'fs/promises';
import * as path from 'path';

type FailureRecord = {
  testFile: string;
  testTitle: string;
  timestamp: string;
  errorFiles: string[];
  dependencies: string[];
  testId: string;
  skipped?: boolean;
};

/**
 * Analyze a test file for Page Object dependencies.
 */
export function analyzePageDependencies(testFilePath: string): string[] {
  const content = fs.readFileSync(testFilePath, 'utf-8');
  const projectRoot = findProjectRoot(testFilePath);
  const deps = new Set<string>();

  for (const p of extractDirectImports(content)) {
    const resolved = resolveImportPath(projectRoot, testFilePath, p);
    if (resolved && isPageObjectPath(resolved)) deps.add(resolved);
  }

  for (const cls of extractPageClassInstantiations(content)) {
    const file = findPageObjectFileByClass(projectRoot, cls);
    if (file) deps.add(toRelative(projectRoot, file));
  }

  return Array.from(deps).sort();
}

/**
 * Return true if any previously failed dependency matches current Page Object deps.
 */
export async function hasFailedPageDependencies(
  dependencies: string[],
): Promise<boolean> {
  if (!dependencies.length) return false;
  const failures = await readDependencyFailures();
  if (!failures.length) return false;

  return failures.some(
    (f) =>
      f.errorFiles?.some((err) => matchesDependency(err, dependencies)) ||
      errorTextContainsDependency(
        [(f as any)?.error?.message ?? '', (f as any)?.error?.stack ?? '']
          .join(' ')
          .toLowerCase(),
        dependencies,
      ),
  );
}

/**
 * Record dependency failure for a test on failure/timeout.
 */
export async function recordPageDependencyFailure(
  testInfo: any,
  dependencies: string[],
): Promise<void> {
  if (!testInfo || !dependencies.length) return;

  const errorFiles = collectErrorFiles(testInfo);
  if (errorFiles.length === 0) return;

  const testId = testInfo.testId || 'unknown';
  const failures = await readDependencyFailures();
  if (failures.find((x) => x?.testId === testId)) return;

  const record: FailureRecord = {
    testFile: testInfo.file || 'unknown',
    testTitle: testInfo.title || 'unknown',
    timestamp: new Date().toISOString(),
    errorFiles,
    dependencies,
    testId,
  };

  failures.push(record);
  await writeDependencyFailures(failures);

  if (Array.isArray(testInfo.annotations)) {
    testInfo.annotations.push({
      type: 'dependency-failure',
      description: "'Recorded Page Object dependency failure',"
    });
  }
}

/**
 * Support util: delete the failures file.
 */
export async function deletePageDependencyFailures(): Promise<void> {
  const p = getFailuresPath();
  await fsp.unlink(p).catch(() => {});
}

/**
 * HELPER: Extract import sources.
 */
function extractDirectImports(content: string): string[] {
  const imports: string[] = [];
  const patterns = [
    /import\s+(?:{[^}]+}|[\w*]+)\s+from\s+['"]([^'\"]+)['"]/g,
    /import\s+['"]([^'\"]+)['"]/g,
    /import\s+\*\s+as\s+\w+\s+from\s+['"]([^'\"]+)['"]/g,
  ];
  for (const re of patterns) {
    let m;
    while ((m = re.exec(content)) !== null) imports.push(m[1]);
  }
  return imports;
}

/**
 * HELPER: Extract class names of `new SomePage(...)`.
 */
function extractPageClassInstantiations(content: string): string[] {
  const classes: string[] = [];
  const re = /new\s+(\w+Page)\s*\(/g;
  let m;
  while ((m = re.exec(content)) !== null) classes.push(m[1]);
  return Array.from(new Set(classes));
}

/**
 * HELPER: Resolve import path relative to the test file and project root.
 */
function resolveImportPath(
  projectRoot: string,
  testFilePath: string,
  importPath: string,
): string | null {
  if (!importPath.startsWith('.') && !importPath.startsWith('/')) return null;
  const testDir = path.dirname(testFilePath);
  const abs = path.resolve(testDir, importPath);

  if (fs.existsSync(abs)) return toRelative(projectRoot, abs);

  const exts = ['.ts', '.tsx', '.js', '.jsx'];
  for (const ext of exts) {
    if (fs.existsSync(abs + ext)) return toRelative(projectRoot, abs + ext);
  }

  const idxs = ['index.ts', 'index.tsx', 'index.js'];
  for (const idx of idxs) {
    const p = path.join(abs, idx);
    if (fs.existsSync(p)) return toRelative(projectRoot, p);
  }

  return null;
}

/**
 * HELPER: Identify Page Object files by convention.
 */
function isPageObjectPath(relPath: string): boolean {
  return (
    relPath.includes('playwright/_redesign/pages') ||
    relPath.includes('playwright/pages') ||
    relPath.endsWith('.page.ts') ||
    relPath.endsWith('.page.tsx')
  );
}

/**
 * HELPER: Find a Page Object file by exported class name.
 */
function findPageObjectFileByClass(
  projectRoot: string,
  className: string,
): string | null {
  const dirs = [
    path.join(projectRoot, 'playwright/_redesign/pages'),
    path.join(projectRoot, 'playwright/pages'),
  ];
  for (const dir of dirs) {
    if (!fs.existsSync(dir)) continue;
    const files = getAllFiles(dir);
    for (const f of files) {
      if (!/\.tsx?$/.test(f)) continue;
      const content = fs.readFileSync(f, 'utf-8');
      if (content.includes(`export class ${className}`)) return f;
    }
  }
  return null;
}

/**
 * HELPER: Recursively list files.
 */
function getAllFiles(dir: string): string[] {
  if (!fs.existsSync(dir)) return [];
  const out: string[] = [];
  for (const entry of fs.readdirSync(dir)) {
    const p = path.join(dir, entry);
    const st = fs.statSync(p);
    if (st.isDirectory()) out.push(...getAllFiles(p));
    else out.push(p);
  }
  return out;
}

/**
 * HELPER: Project root discovery.
 */
function findProjectRoot(startPath: string): string {
  let cur = path.resolve(startPath);
  while (cur !== path.dirname(cur)) {
    const pkg = path.join(cur, 'package.json');
    if (fs.existsSync(pkg)) return cur;
    cur = path.dirname(cur);
  }
  return cur;
}

/**
 * HELPER: Normalize to project-relative unix path.
 */
function toRelative(root: string, abs: string): string {
  return path.relative(root, abs).replace(/\\/g, '/');
}

/**
 * Persist/Load failures.
 */
function getFailuresPath(): string {
  const tmpDir = path.join(process.cwd(), 'tmp');
  return path.join(tmpDir, 'dependency-failures.json');
}

async function readDependencyFailures(): Promise<FailureRecord[]> {
  try {
    const p = getFailuresPath();
    const txt = await fsp.readFile(p, 'utf-8').catch(() => '[]');
    if (!txt.trim()) return [];
    const j = JSON.parse(txt);
    return Array.isArray(j) ? j : [];
  } catch {
    return [];
  }
}

async function writeDependencyFailures(
  failures: FailureRecord[],
): Promise<void> {
  const tmpDir = path.join(process.cwd(), 'tmp');
  await fsp.mkdir(tmpDir, { recursive: true });
  const p = getFailuresPath();
  await fsp.writeFile(p, JSON.stringify(failures, null, 2));
}

/**
 * Error helpers.
 */
function collectErrorFiles(testInfo: any): string[] {
  const set = new Set<string>();
  const addFrom = (stack?: string) => {
    if (!stack) return;
    const re = /at\s+.*?\((.+?):\d+:\d+\)|at\s+(.+?):\d+:\d+/g;
    let m;
    const root = process.cwd();
    while ((m = re.exec(stack)) !== null) {
      const file = (m[1] || m[2])?.replace(/\\/g, '/');
      if (!file) continue;
      if (file.includes('node_modules')) continue;
      const abs = path.isAbsolute(file)
        ? file
        : path.resolve(root, file).replace(/\\/g, '/');
      set.add(abs);
    }
  };

  const stepMap = (testInfo as any)?._stepMap;
  if (stepMap) {
    for (const [, step] of stepMap.entries()) {
      if (step?.error && step?.location?.file) set.add(step.location.file);
      if (step?.error?.stack) addFrom(step.error.stack);
    }
  }
  if (testInfo.error?.location?.file) set.add(testInfo.error.location.file);
  if (testInfo.error?.stack) addFrom(testInfo.error.stack);
  if (testInfo.result?.error?.location?.file)
    set.add(testInfo.result.error.location.file);
  if (testInfo.result?.error?.stack) addFrom(testInfo.result.error.stack);
  for (const err of testInfo.errors || []) {
    if (err?.location?.file) set.add(err.location.file);
    if (err?.stack) addFrom(err.stack);
  }
  return Array.from(set);
}

function errorTextContainsDependency(
  lowerText: string,
  dependencies: string[],
): boolean {
  return dependencies.some((dep) => {
    const base = dep.split('/').pop() || dep;
    const parts = dep.split('/').filter(Boolean);
    const lowerDep = dep.toLowerCase();
    return (
      lowerText.includes(base.toLowerCase()) ||
      lowerText.includes(lowerDep) ||
      parts.some((p) => lowerText.includes(p.toLowerCase()))
    );
  });
}

function matchesDependency(errorFile: string, dependencies: string[]): boolean {
  if (!errorFile) return false;
  const normalized = errorFile.replace(/^.*monument-core\//, '');
  const base = normalized.split('/').pop() || '';
  return dependencies.some((dep) => {
    const depBase = dep.split('/').pop() || dep;
    return (
      normalized.includes(dep) ||
      errorFile.includes(dep) ||
      depBase === base ||
      normalized.endsWith(dep)
    );
  });
}

---

Step 2: Wire the Analyzer into a Fixture

Now we'll use this analyzer in our Playwright setup. The best place is by extending the context fixture, as it runs before each test. We'll import the functions we just built and inject the logic.

Create a new fixture file (or add to your existing one):

import { test as base } from '@playwright/test';
import {
  analyzePageDependencies,
  hasFailedPageDependencies,
  recordPageDependencyFailure,
} from './testDependencyAnalyzer'; // Adjust the import path

export const test = base.extend({
  // We override the 'context' fixture.
  context: async ({ browser, baseURL }, use) => {
    // 1. Get info for the test that is about to run
    const testInfo = base.info();

    // 2. ANALYZE: Get Page Object dependencies for the current test file
    const deps = analyzePageDependencies(testInfo.file);

    // 3. CHECK & SKIP: Check if any of those dependencies have failed before
    if (await hasFailedPageDependencies(deps)) {
      // If yes, skip this test immediately
      test.skip(
        true,
        'Skipping due to previous Page Object dependency failures',
      );
    }

    // --- Standard fixture setup ---
    const context = await browser.newContext({
      baseURL,
      ignoreHTTPSErrors: true,
    });
    // --- End standard setup ---

    try {
      // 4. RUN: Run the test
      await use(context);
    } finally {
      // 5. RECORD: This block runs after the test is finished
      await context.close();

      // If the test failed or timed out, record its dependencies as failed
      if (testInfo.status === 'failed' || testInfo.status === 'timedOut') {
        await recordPageDependencyFailure(testInfo, deps);
      }
    }
  },
});

// Re-export 'expect' so users can import { test, expect } from this file
export { expect } from '@playwright/test';

How It Works: The Fixture Logic

1. Get Test Info: We get the test's file path from testInfo.file.
2. Analyze: We pass this path to our new analyzePageDependencies function.
3. Check & Skip: We pass the resulting dependencies to hasFailedPageDependencies. If it returns true, we call test.skip() and the test is never executed.
4. Run: If it returns false, the test runs as normal using await use(context).
5. Record: In the finally block, we check testInfo.status. If the test failed, we call recordPageDependencyFailure to log the failure, which will cause dependent tests to be skipped in the next run.

---

Step 3: Usage in Tests

This is the best part. You don't need to change your tests at all.

The only change is the import. Instead of importing test from @playwright/test, you import it from your new fixture file.

Before:
import { test, expect } from '@playwright/test';

After:
import { test, expect } from './my-fixtures.ts'; // (or wherever you saved your new fixture)

The fixture handles all the analysis and skip logic automatically.

---

Step 4: Clearing the Failure Cache

The tmp/dependency-failures.json file will persist between runs. This is intentional.

However, you'll want to clear it before starting a fresh CI run. You can use the deletePageDependencyFailures helper we built.

The easiest way is to add a globalSetup to your playwright.config.ts:

// playwright.config.ts
import { defineConfig } from '@playwright/test';
import { deletePageDependencyFailures } from './playwright/utils/testDependencyAnalyzer';

export default defineConfig({
  // ...
  globalSetup: async () => {
    // Clear the dependency failure cache before a full run
    await deletePageDependencyFailures();
  },
  // ...
});

Conclusion

You now have a powerful, lightweight test dependency system. It's built to analyze your Page Object dependencies to create a smart, self-healing test execution flow.

This approach gives you smarter test execution and cleaner failure reports by skipping tests that are destined to fail, all while remaining simple and decoupled from your test logic.