DEV Community: KRISHNA S

Debug Playwright failures and run history without leaving VS Code

KRISHNA S — Wed, 07 Oct 2026 19:04:17 +0000

You have a failure open in the Playwright HTML report. You read the exception, switch to your editor to inspect the code, then return to the report to check another attempt or earlier result.

I wanted to make that investigation easier to do where the code already is.

Playwright Logbook for VS Code brings the Logbook investigation workflow into the IDE: Recent Runs → run overview → test details → attempts and history → source. Analyze with AI also prepares a failure-analysis task for your chosen coding assistant chat.

The open-source Logbook reporter saves the run evidence and history; the extension brings those records beside your code. New to the reporter? Start with A Playwright reporter that remembers your runs and briefs your coding assistant.

Open Logbook: 🧩 VS Code Marketplace · 📦 Reporter on npm · ⭐ GitHub repository

Use the reporter to collect your usual local runs. Then browse their summaries, inspect errors and available diagnostics, compare recorded executions, and open source without repeatedly switching between the report and editor.

Get a useful first run in a few steps

Install the extension:

code --install-extension krishnapollu.playwright-logbook-vscode

Or search for Playwright Logbook, publisher krishnapollu, in VS Code.

The extension reads history collected by the Logbook reporter. Add that reporter to your existing project:

npm install --save-dev playwright-logbook

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['list'],
    // Opt-in: capture steps, output tails and eligible image previews.
    ['playwright-logbook', { captureDetails: true }],
  ],
  use: {
    screenshot: 'only-on-failure',
    trace: 'on-first-retry',
  },
});

captureDetails defaults to false; this example opts in to extra diagnostics. trace: 'on-first-retry' captures a trace only when a first retry occurs—it does not enable retries. Keep your existing retry policy, or use --retries=1 when trying this on a demo suite.

npx playwright test

Open Logbook in the activity bar. Keep .logbook/runs/ and .logbook/index.jsonl between runs so you have history to inspect.

If your Playwright project lives in a subfolder, set the history and source paths first (see A couple of setup details worth getting right below).

The extension supports desktop VS Code 1.95+ on macOS, Windows and Linux in local workspaces only (no Remote SSH, containers or browser-based VS Code). The reporter requires Node.js 20+ and Playwright 1.42+.

Browse runs from the sidebar

The native Recent Runs sidebar is your starting point. Expand a saved run to see its results or choose Run overview when you need the whole-run picture.

Click any screenshot to inspect the original image at full resolution.

The native tree keeps the run and its recorded results within reach while you work in VS Code.

The overview shows recorded outcome totals, project breakdowns, a clickable cases list and run-level errors when recorded. Select a case to investigate it; the sidebar search action helps find a test in a larger run.

You can review a saved run without executing the suite again.

The selected pw-test run has 16 passes, two deliberate failures and two skips; the cases list links into individual results.

Filter to the tests you are working on

Use the live filter to narrow saved runs and results by test name, spec path, project, status, branch, commit or other recorded metadata. Search older runs extends the loaded history; Clear resets the filter.

You can also start from your code: right-click a spec or test file in Explorer to filter to that file, or right-click inside a test and choose Logbook: Filter This Test. The sidebar then shows the relevant runs and results.

Filter directly from the test you are inspecting in the editor.

Keep test diagnostics beside your code

Select a test to see its recorded outcome, error, assertion details and source excerpt. The test view keeps the available Steps, Logs, Errors and Attachments close to its attempts and execution history.

Use Open failure location to inspect the relevant code, then return to the same result to review the diagnostic. You can also open the recorded test definition. Historical line numbers may have moved in your current checkout.

That matters when an initial attempt fails but a retry succeeds. The final outcome is useful, but the earlier diagnostic may tell you what to investigate next.

captureDetails: true enables optional diagnostic capture for new runs. Eligible bounded PNG/JPEG images can be previewed; missing capture data stays unavailable. Existing runs cannot acquire evidence retrospectively.

A deliberate assertion failure keeps its code frame, retry summary, source actions and matching test history together.

Analyze the failure with your coding assistant

Analyze with AI prepares an unsent task in your chosen assistant chat, so you can ask for analysis without assembling the context by hand. Review the task and evidence for secrets before submitting.

Open a test result and choose ✧ Analyze with AI ▾ beside the source actions.
On first use, pick an installed coding assistant or native VS Code Chat. Logbook remembers your choice for that workspace; use the arrow to change it.
In a supported assistant, review the prepared unsent draft, choose your model, and submit it. Read the analysis and continue the investigation in that assistant's chat.

The task points to saved failure context: the selected execution, errors, attempts, captured steps and output, attachment references, matching history, and a bounded excerpt of current test source when available. The prompt asks for a concise analysis with a likely cause, supporting evidence and next steps. Current source may differ from the recorded run, and missing diagnostics remain missing.

In the current release, supported handoffs include native VS Code Chat, Codex, Antigravity, Amazon Q and Cline. Analysis requires Workspace Trust and access to your chosen assistant. If an assistant's input integration is unsupported, Logbook reports it so you can choose another.

Logbook prepares the handoff; it does not submit the request or read the answer. Submitting sends the task, and any evidence the assistant reads, to that assistant and whichever model provider it uses, under that tool’s settings and policies. Text filtering does not guarantee that every secret is removed and cannot remove secrets visible in screenshots.

The action sits beside the source controls. Its arrow changes the saved assistant choice.

Compare the test across recorded executions

The History panel lets you select another recorded execution of the matching test/project or compare it with the selected result.

The comparison brings together outcomes, available errors, attempts and labeled durations, so you can inspect the difference between two recorded executions within the test workflow. Recorded branch and commit information supplies context when present.

Start with an earlier local run of the same test. A useful comparison might reveal that the final outcome stayed failed but the recorded error changed, or that a pass required more attempts. Those are observations to investigate, rather than an automatic diagnosis.

The useful question becomes more specific:

Did the same test fail in an earlier execution, and how do the recorded attempts and errors differ?

History is limited to available records and the selected scope. Matching error text does not establish a shared root cause, and one successful retry does not prove a test is now reliable.

Two executions of the same pw-test case both failed with the same recorded message. The comparison still exposes their run identities, attempts and durations.

Inspect source changes with native Git diffs

When recorded revisions are available locally, the comparison shows a committed test-file diff beside the recorded outcome changes. Open either revision’s source or choose View full file diff for the native Git diff. Git actions require Workspace Trust and local Git.

No fetch or checkout is performed. A recorded commit shows committed content; it cannot reconstruct uncommitted changes that were present during execution. The extension keeps that limitation visible rather than presenting code changes as a confirmed failure cause.

Recorded outcome changes and committed assertion edits appear together, with actions to open the full diff or either revision’s source.

An added option: import a CI run bundle

The same workflow can also include a recorded CI execution. With reporter 0.3.0+, export a saved run as a portable bundle and download it locally:

In CI, after the run is saved and after merging shards if your jobs are sharded:

npx playwright-logbook export \
  --artifacts \
  --out ci-investigation.logbook.zip \
  --project-id checkout-tests

After downloading the bundle:

Choose Logbook: Import Run Bundle… from the Command Palette or sidebar.
Select the ZIP and use the same stable project ID as CI.
Review the validation results, then choose Import and Open Imported Run.

Imported results join local history when test and project identities match; branch scope still applies. Duplicate/conflicting records are checked. Renamed tests are not automatically reconciled.

This is a local, reviewed import. The extension does not automatically download CI artifacts. For sharded jobs, merge the run before exporting its bundle.

The review reports what the bundle would add before import. This example has zero new runs, one identical run and six missing or omitted artifact references; the capture was taken without completing the import.

A couple of setup details worth getting right

If your Playwright project is nested inside a repository, map the history and source roots:

{
  "logbook.historyPath": "e2e/.logbook",
  "logbook.sourceRoot": "e2e"
}

Select the history folder containing index.jsonl and runs/, not just the HTML report.

Imported trace/video files can retain their evidence and metadata, but embedded trace viewing and video playback are not available. Full screen-reader validation is ongoing.

Fits alongside your existing tools

Keep using Playwright's official extension to run, generate and debug tests, and its Trace Viewer for detailed execution inspection. Logbook adds a saved-history investigation workflow and an optional handoff to your coding assistant. You review and submit the AI task in that assistant’s chat.

You can try it on a small suite first: collect a few runs, inspect an earlier failure, and see whether the trip from evidence to source is easier.

Try Logbook on a suite you already investigate. Which part would help most: clearer exceptions and attempts, run triage, history, or a better handoff to your coding assistant? Share your thoughts, feature ideas or experience in the comments, or open an issue. If it helps your workflow, a ⭐ on GitHub helps other Playwright users find it.

Topics: #testautomation · #devtools · #opensource · #ai · #playwright · #reporting · #debugging · #playwright-logbook

A Playwright reporter that remembers your runs and briefs your coding assistant

KRISHNA S — Mon, 05 Oct 2026 23:21:43 +0000

A red test result starts an investigation. Which attempt failed? What did the assertion actually say? Was this test failing on earlier runs? Where is the evidence you would hand to a teammate or coding assistant?

Playwright Logbook is an open-source reporter that keeps failure diagnostics, retries, run-to-run changes and history in one local HTML report. Its CLI can also give a coding assistant a focused, reviewable packet of recorded evidence.

Try Logbook: 📦 Reporter on npm · 🧩 VS Code extension · ⭐ GitHub repository

pw-test is a demo project. This saved run has 20 results across API and Chromium projects, including two failing API tests. These are sample diagnostics, not a production incident or a performance benchmark. Click a screenshot to inspect its original resolution. Screenshots come from saved demo runs and may differ between tabs.

Playwright's HTML report and Trace Viewer are useful. Logbook focuses on the next steps: bringing the exception, attempts, available artifacts, saved-run changes and a debug packet with source excerpts into one workflow.

Try it in your Playwright project

npm install --save-dev playwright-logbook

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['list'],
    // Opt-in: capture steps, output tails and eligible image previews.
    ['playwright-logbook', { captureDetails: true }],
  ],
  use: {
    screenshot: 'only-on-failure',
    trace: 'on-first-retry',
  },
});

captureDetails defaults to false. The example opts in to extra diagnostics. trace: 'on-first-retry' captures a trace only if a first retry occurs; it does not enable retries. Keep your existing retry policy, or use --retries=1 when trying this on a demo suite.

Run npx playwright test, then open .logbook/report/index.html. The single HTML file opens locally without a server. Keep .logbook/runs/ and .logbook/index.jsonl to build history. The reporter requires Node.js 20+ and Playwright 1.42+.

Give your coding assistant the recorded evidence

A pasted error is only part of the story. For a quick handoff, open the test in the HTML detail panel, use Preview debug context, review it for secrets, then Copy debug context to share the recorded evidence with your coding assistant.

For an assistant with terminal and file access to your project, the CLI offers a debug packet for one test in one run:

npx playwright-logbook history --limit 10 --json
npx playwright-logbook summary --run latest --format json
npx playwright-logbook debug --run latest --test YOUR_TEST_ID --format json

Find the recorded testId in .logbook/runs/<run-id>.json; the summary gives aggregate counts, not a list of every failed test. The debug packet includes available attempt outcomes, errors, source excerpts, steps and output, attachment metadata, recent outcomes, evidence IDs and a rerun command. Ask for the same test ID from an earlier run to compare the two packets. There is no separate CLI diff command.

Logbook does not call an AI provider or fix tests automatically. Missing evidence is called out rather than invented. Review packets and attachments before sharing: text redaction does not guarantee that every secret is removed, and it cannot remove secrets visible in screenshots.

Passed test opened in the test detail window which shows the test steps, run history, logs, attachments and option to compare with previous runs

Investigate the exact failure

The Tests view filters by status, project and tag, and groups by file. Open a result for its recorded error, code frame and expandable stack trace. Switch between attempts to see what failed before a retry, with optional captured steps, output and eligible image previews when captureDetails: true was enabled for that run.

Attachments show what was retained. A trace can be opened with Playwright's Trace Viewer through a copied command; Logbook does not embed a trace viewer.

Failed test in the test detail window with error context, steps, attachments, logs and option to compare with previous runs.

Triage the run, then look back

The Failures tab groups similar recorded errors. It also identifies new failures, fixed tests, tests still failing, new tests and removed tests relative to the previous saved run. These are triage clues, not a confirmed common cause.

Logbook Failures tab grouping two failing API tests under one error message, with run-change buckets showing two still failing

The Trends view and per-test recent-results strip add context from retained runs. A failure today may be new, recurring or a retry recovery; those lead to different questions. History only reflects executions you kept and cannot establish root cause.

Retained pw-test runs give the trend its context.

The Run tab adds recorded environment details and an approximate worker timeline for unsharded runs when timing data supports it.

Recorded environment and shard details, with an approximate worker timeline.

The Project tab breaks results down by Playwright project and file.

API and Chromium results, followed by the breakdown by test file.

Bring the investigation into your editor

Logbook sidebar and run overview in VS Code, showing saved runs and the cases list

The VS Code companion browses those saved results beside your code: run overview, failure detail, history and recorded source. It also provides execution comparison in its UI. With the CLI, you ask your assistant to compare two debug packets. The reporter remains responsible for collection.

You can also export a portable Logbook bundle from CI, review it and import it into local history. This is a manual workflow, with no automatic CI download. The bundle guide covers project IDs, sharded runs, artifacts and validation.

Prefer to stay in your editor? See Debug Playwright failures and run history without leaving VS Code.

Exploring WebdriverIO

KRISHNA S — Tue, 29 Aug 2023 02:45:45 +0000

Hello QA's,

I recently bumped into WebdriverIO during my search for a better UI automation tool. Being into Selenium+Java, I have always found it to be hectic building frameworks, implementing listeners / hooks and wrappers, writing custom reporters / loggers and what not. But this new NodeJS based tool ( in fact, almost all the new age tools including Cypress and Playwright ) has made things simpler; be it the setup or integration with third party tools. I wanted to share with you what I found catchy with this particular tool. The points below might be applicable to others as well but for now let's focus on WebdriverIO.

Setup - All you need to have as pre requisite is a node installation and an IDE. The npm will do the rest for you.
Configuration - The whole bulky framework is now reduced to a single config.js file. You just command the WebdriverIO "Hey! Run the tests with chrome." Or "I want you to generate an allure report for my test". And that's it. WebdriverIO does it for you. The reporters, logging, hooks, drivers, test framework; all these are mentioned in the conf.js file.
Integrate any kind of tests you wish to; be it Web UI or Mobile App UI or even API/ webservices calls. This is one of the features which made me pick WebdriverIO. I work on projects that has test scenarios integrated with Web + Mobile App + REST API. And most of the UI automation tools out there doesn't have this capability of integrating mobile app testing. WebdriverIO lets you achieve this by integrating Appium to your test suite.
Run your tests anywhere without touching your code. WebdriverIO executes your tests in local env as well as any cloud platforms ( saucelabs, browserstack etc. ). You just mention it in your config.js and leave the rest to WebdriverIO.
Multiremote - a feature that helps you handle multiple browser / driver sessions within the same worker. You can use this for cross browser testing, a server client app testing or a web + mobile app integrated testing.
Multiple test frameworks support - mocha, jasmine and cucumber. I was able to maintain two different test suites within the same project - cucumber based suite for e2e tests and mocha based suite for data driven tests.
Features like Isolated test contexts, auto-waiting, own assertion library, component testing.
And a detailed documentation.

But still I feel few things are missing in WebdriverIO which would have made my life easier.

Better debug options
Better ways to handle flaky tests
A code generator for recording the tests

Here is one sample project I have done. The WebdriverIO team was kind enough to list this among their boilerplate projects.

WebdriverIO - Boilerplate Projects

Github