A Playwright reporter that remembers your runs and briefs your coding assistant

KRISHNA S — Mon, 05 Oct 2026 23:21:43 +0000

A red test result starts an investigation. Which attempt failed? What did the assertion actually say? Was this test failing on earlier runs? Where is the evidence you would hand to a teammate or coding assistant?

Playwright Logbook is an open-source reporter that keeps failure diagnostics, retries, run-to-run changes and history in one local HTML report. Its CLI can also give a coding assistant a focused, reviewable packet of recorded evidence.

Try Logbook: 📦 Reporter on npm · 🧩 VS Code extension · ⭐ GitHub repository

pw-test is a demo project. This saved run has 20 results across API and Chromium projects, including two failing API tests. These are sample diagnostics, not a production incident or a performance benchmark. Click a screenshot to inspect its original resolution. Screenshots come from saved demo runs and may differ between tabs.

Playwright's HTML report and Trace Viewer are useful. Logbook focuses on the next steps: bringing the exception, attempts, available artifacts, saved-run changes and a debug packet with source excerpts into one workflow.

Try it in your Playwright project

npm install --save-dev playwright-logbook

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['list'],
    // Opt-in: capture steps, output tails and eligible image previews.
    ['playwright-logbook', { captureDetails: true }],
  ],
  use: {
    screenshot: 'only-on-failure',
    trace: 'on-first-retry',
  },
});

captureDetails defaults to false. The example opts in to extra diagnostics. trace: 'on-first-retry' captures a trace only if a first retry occurs; it does not enable retries. Keep your existing retry policy, or use --retries=1 when trying this on a demo suite.

Run npx playwright test, then open .logbook/report/index.html. The single HTML file opens locally without a server. Keep .logbook/runs/ and .logbook/index.jsonl to build history. The reporter requires Node.js 20+ and Playwright 1.42+.

Give your coding assistant the recorded evidence

A pasted error is only part of the story. For a quick handoff, open the test in the HTML detail panel, use Preview debug context, review it for secrets, then Copy debug context to share the recorded evidence with your coding assistant.

For an assistant with terminal and file access to your project, the CLI offers a debug packet for one test in one run:

npx playwright-logbook history --limit 10 --json
npx playwright-logbook summary --run latest --format json
npx playwright-logbook debug --run latest --test YOUR_TEST_ID --format json

Find the recorded testId in .logbook/runs/<run-id>.json; the summary gives aggregate counts, not a list of every failed test. The debug packet includes available attempt outcomes, errors, source excerpts, steps and output, attachment metadata, recent outcomes, evidence IDs and a rerun command. Ask for the same test ID from an earlier run to compare the two packets. There is no separate CLI diff command.

Logbook does not call an AI provider or fix tests automatically. Missing evidence is called out rather than invented. Review packets and attachments before sharing: text redaction does not guarantee that every secret is removed, and it cannot remove secrets visible in screenshots.

Passed Chromium test opened in the Logbook detail panel with rerun command, debug context and steps

Investigate the exact failure

The Tests view filters by status, project and tag, and groups by file. Open a result for its recorded error, code frame and expandable stack trace. Switch between attempts to see what failed before a retry, with optional captured steps, output and eligible image previews when captureDetails: true was enabled for that run.

Attachments show what was retained. A trace can be opened with Playwright's Trace Viewer through a copied command; Logbook does not embed a trace viewer.

Failed test in the Logbook detail panel with error context, steps and option to switch between attempts.

Triage the run, then look back

The Failures tab groups similar recorded errors. It also identifies new failures, fixed tests, tests still failing, new tests and removed tests relative to the previous saved run. These are triage clues, not a confirmed common cause.

Logbook Failures tab grouping two failing API tests under one error message, with run-change buckets showing two still failing

The Trends view and per-test recent-results strip add context from retained runs. A failure today may be new, recurring or a retry recovery; those lead to different questions. History only reflects executions you kept and cannot establish root cause.

Retained pw-test runs give the trend its context.

The Run tab adds recorded environment details and an approximate worker timeline for unsharded runs when timing data supports it.

Recorded environment and shard details, with an approximate worker timeline.

The Project tab breaks results down by Playwright project and file.

API and Chromium results, followed by the breakdown by test file.

Bring the investigation into your editor

Logbook sidebar and run overview in VS Code, showing saved runs and the cases list

The VS Code companion browses those saved results beside your code: run overview, failure detail, history and recorded source. It also provides execution comparison in its UI. With the CLI, you ask your assistant to compare two debug packets. The reporter remains responsible for collection.

You can also export a portable Logbook bundle from CI, review it and import it into local history. This is a manual workflow, with no automatic CI download. The bundle guide covers project IDs, sharded runs, artifacts and validation.

Try Logbook on a suite you already investigate. Which part would help most: clearer exceptions and attempts, run triage, history, or a better handoff to your coding assistant? Share your thoughts, feature ideas or experience in the comments, or open an issue. If it helps your workflow, a ⭐ on GitHub helps other Playwright users find it.

Exploring WebdriverIO

KRISHNA S — Tue, 29 Aug 2023 02:45:45 +0000

Hello QA's,

I recently bumped into WebdriverIO during my search for a better UI automation tool. Being into Selenium+Java, I have always found it to be hectic building frameworks, implementing listeners / hooks and wrappers, writing custom reporters / loggers and what not. But this new NodeJS based tool ( in fact, almost all the new age tools including Cypress and Playwright ) has made things simpler; be it the setup or integration with third party tools. I wanted to share with you what I found catchy with this particular tool. The points below might be applicable to others as well but for now let's focus on WebdriverIO.

Setup - All you need to have as pre requisite is a node installation and an IDE. The npm will do the rest for you.
Configuration - The whole bulky framework is now reduced to a single config.js file. You just command the WebdriverIO "Hey! Run the tests with chrome." Or "I want you to generate an allure report for my test". And that's it. WebdriverIO does it for you. The reporters, logging, hooks, drivers, test framework; all these are mentioned in the conf.js file.
Integrate any kind of tests you wish to; be it Web UI or Mobile App UI or even API/ webservices calls. This is one of the features which made me pick WebdriverIO. I work on projects that has test scenarios integrated with Web + Mobile App + REST API. And most of the UI automation tools out there doesn't have this capability of integrating mobile app testing. WebdriverIO lets you achieve this by integrating Appium to your test suite.
Run your tests anywhere without touching your code. WebdriverIO executes your tests in local env as well as any cloud platforms ( saucelabs, browserstack etc. ). You just mention it in your config.js and leave the rest to WebdriverIO.
Multiremote - a feature that helps you handle multiple browser / driver sessions within the same worker. You can use this for cross browser testing, a server client app testing or a web + mobile app integrated testing.
Multiple test frameworks support - mocha, jasmine and cucumber. I was able to maintain two different test suites within the same project - cucumber based suite for e2e tests and mocha based suite for data driven tests.
Features like Isolated test contexts, auto-waiting, own assertion library, component testing.
And a detailed documentation.

But still I feel few things are missing in WebdriverIO which would have made my life easier.

Better debug options
Better ways to handle flaky tests
A code generator for recording the tests

Here is one sample project I have done. The WebdriverIO team was kind enough to list this among their boilerplate projects.

WebdriverIO - Boilerplate Projects

Github