Playwright Debugging, Supercharged by GitHub Copilot

My Portfolio :(https://shekhar-pro.vercel.app/)

What I Built

I built TestPilot AI, an AI-augmented Playwright test runner that automatically diagnoses failing tests and suggests fixes using the GitHub Copilot SDK and CLI.

The project’s main goal is to reduce the time developers spend debugging failing tests. Instead of manually inspecting error messages or stack traces, TestPilot AI sends failure details to Copilot, which analyzes the failure, identifies the root cause, classifies the issue (e.g., flaky test, locator change, API failure), and even provides a suggested code fix or improved test snippet.

For me, this project demonstrates how Copilot can act as a real development partner — not just generating code, but actively helping debug and optimize tests in real time.

Demo

Project Repository: (https://github.com/jancsp3/GHCPCLI-Shekhar)

Video Walkthrough:(https://youtu.be/JB_DcMVHVh8)

Run the AI test runner:

npm run test

A test fails intentionally.

Copilot SDK automatically analyzes the failure and prints a detailed diagnostic report in the terminal, including:

Root cause

Suggested code fix

Updated test snippet

Diagnostic report is also saved in /ai-diagnostics for review.

Example screenshot / snippet:

AI analyzing failure in: homepage should have correct title

--- COPILOT DIAGNOSTIC REPORT ---
Root Cause: The test expected 'Wrong Title' but the actual page title is 'Example Domain'.
Suggested Fix: Update the test to expect the correct title.
Improved Test Snippet:
await expect(page).toHaveTitle('Example Domain');

---

My Experience with GitHub Copilot CLI

I used GitHub Copilot CLI throughout the project to accelerate development and ensure high-quality outputs:

Project scaffolding:

gh copilot suggest "Create a Playwright TypeScript project structure with tests folder and custom fixtures"

Copilot generated a ready-to-use project structure, saving me time on setup.

AI diagnostic fixture creation:

gh copilot suggest "Create a Playwright fixture that runs after test failure and sends error details to Copilot SDK for analysis"

I refined the generated code to handle error reporting and session cleanup.

Prompt optimization:

gh copilot suggest "Improve this AI prompt for diagnosing Playwright test failures"

This helped produce more accurate diagnostic outputs.

Debugging and refactoring:

gh copilot explain ""
gh copilot suggest "Improve error handling in Playwright fixture"

Copilot provided explanations and suggested refactors, making iteration much faster.

Overall, Copilot CLI acted like a collaborative coding assistant — not only generating code but guiding me to improve prompts, debug errors, and optimize test workflows. It significantly reduced development time and made the testing process more reliable and intelligent.