DEV Community: TestDino

TestDino Public API: One Call for Errors, Coverage, Specs, and Traces. Here Is What That Enables.

TestDino — Mon, 01 Jun 2026 09:18:37 +0000

If you have ever tried to get Playwright test data out of a reporting tool programmatically, you know the problem. You call the list endpoint, get statuses, then make a separate call per run for errors, another for coverage, another for spec breakdown. By the time you have enough data to post something useful to a PR or a CI pipeline export, you have chained four requests and written more glue code than the feature itself.

The TestDino Public API handles this differently. One parameter changes what you get back from a single run detail call. That difference is what makes building a Playwright reporting API integration actually worth the time.

Note: TestDino is Playwright-only. Cypress, Vitest, Jest: not supported.

The ?include= Parameter

Default call to a run detail endpoint returns run metadata: status, duration, branch, environment, test counts. Lean and fast.

Add ?include=errors,coverage,specs,artifacts and the same request returns:

errors: failures grouped by error message with stack traces
coverage: Istanbul code coverage metrics for that run
specs: spec files with flattened test cases and individual status per test
artifacts: authenticated download URLs for Playwright traces, screenshots, videos

One call. No chaining.

GET /{projectId}/test-runs/{runId} include=errors,coverage,specs,artifacts

The ?include= parameter only works on two endpoints. GET /{projectId}/test-runs/{runId} accepts errors, coverage,specs,artifacts, all. GET /{projectId}/test-cases/{caseId} accepts history, all. List endpoints, analytics, and dashboard endpoints do not support it.

Full Endpoint List

Base URL:

https://api.testdino.com/api/public/v1/{projectId}/

Auth: Bearer token with tdp_ prefix, public-api scope. Generate from Project Settings. Token is scoped to one project. Project A token returns 403 on project B.

Rate limits: 100 req/min per token, 200/min per IP. Every response includes RateLimit-Remaining and RateLimit-Reset. PDF generation is 1/min per token.

GET  /token-info                 # verify PAT, read project and org metadata
GET  /test-runs                  # list runs, filter by branch/env/developer/tag/date
GET  /test-runs/{runId}          # run detail + ?include=errors,coverage,specs,artifacts
GET  /test-cases/{caseId}        # test case detail + ?include=history
GET  /specs                      # spec file health: pass rate, flakiness score, duration
GET  /test-case-explorer         # aggregated metrics across runs
GET  /analytics/summary          # summary metrics and trend data
GET  /dashboard                  # project health overview
GET  /manual-tests               # manual suites, cases, steps
POST /reports/pdf                # generate a PDF report
GET  /filters                    # environments, branches, developers, tags in a project
GET  /usage                      # subscription usage and allocation

Full auth and error handling: docs.testdino.com/api-reference/conventions
Five-minute quickstart: docs.testdino.com/api-reference/quickstart

What You Can Actually Build

1. GitHub PR Annotation

The pain before: Reviewers merging without knowing if tests passed. The "can you check" comment in every PR thread.

After CI finishes, a workflow step calls the latest run for that branch, pulls pass rate and top failed test names with errors via ?include=errors, posts a PR comment. Reviewer sees test status without leaving GitHub.

GET /test-runs?branch=feature/my-branch&limit=1
GET /test-runs/{runId}?include=errors

2. Nightly Digest Email

The pain before: Engineering leadership has no visibility into test health trends unless someone manually checks and reports.

A scheduled script paginates through the previous day's runs, aggregates pass rate and failure counts per branch, grabs the five most flaky spec files via GET /specs, formats a plain text email, sends it. Director reads it before standup. Automated. Nobody had to remember.

3. Client-Facing Status Page

The pain before: Clients emailing "what is the current test status" weekly.

A script pulls the latest run and writes pass rate, timestamp, and branch to a Notion database or simple webpage. Clients check it themselves. No TestDino account required on their end. Those messages stop.

4. Flakiness Heat Map

The pain before: Knowing a suite is getting unstable but not knowing which files are pulling the numbers down.

GET /specs returns per-file flakiness scores across runs. GET /test-cases/{caseId}?include=history&historyStatus=flaky gives per-test flaky history. Combine them into a weekly view that surfaces exactly which files and tests are degrading. Teams that build this catch the problem before it becomes a standup conversation.

5. MCP: Querying Test Data from an AI Tool

The TestDino MCP server is a separate access path from the REST API. Two versions:

A. Local MCP Install via npx, connect Cursor, Claude Code, or Claude Desktop. The IDE tool queries live run data directly during your session.

npx @testdino/mcp

Local MCP setup

B. Remote MCP Hosted at mcp.testdino.com, connects ChatGPT, Claude on the web, and other remote-MCP-capable tools. No local install.

Remote MCP setup

The pain before: Copying stack traces into an AI prompt and hoping it reasons correctly from pasted context.

The practical shift with MCP: You ask "what error pattern appeared most in staging this week" and the assistant queries actual run data to answer. Not a guess. Not training data. Your data.

Based on the public alternatives comparison, no other Playwright reporting tool has a remote MCP server.

Access and Availability

The API and MCP are included on every plan. There is no separate API tier, no add-on, and no unlock. You can start on the free plan and make your first API call today without a credit card.

Most alternatives in this space gate their API behind paid plans or charge significantly more for equivalent access. Full breakdown on the alternatives comparison page.

Get started | Pricing

Mistake 14/14: Not using the tools Playwright already gives you

TestDino — Tue, 21 Apr 2026 13:30:37 +0000

This is the final mistake in the series.

A lot of teams still debug failed tests the hard way:

read the error
add a console log
re-run the suite
repeat

That works, but it is slow.

Playwright already ships with tools that make debugging much easier:

Trace Viewer

Replay a failed run step by step and inspect DOM state, actions, network calls, and timing.

npx playwright show-trace trace.zip

UI Mode

Run tests in an interactive mode and watch them execute with time-travel debugging.

npx playwright test --ui

Codegen

Generate locators and actions by recording browser interactions.

npx playwright codegen https://your-app.com

Last failed

Re-run only the tests that failed, not the whole suite.

npx playwright test --last-failed

Use scripts

You can also add these to package.json so the team actually uses them regularly.

{
  "scripts": {
    "test": "npx playwright test",
    "test:ui": "npx playwright test --ui",
    "test:failed": "npx playwright test --last-failed",
    "test:trace": "npx playwright show-trace trace.zip",
    "codegen": "npx playwright codegen"
  }
}

The biggest lesson from this series is simple: Most debugging pain is not caused by missing tools. It is caused by not using the tools you already have.

Playwright Mistake 13/14: You have no test cleanup strategy

TestDino — Fri, 17 Apr 2026 13:09:08 +0000

Your tests create users, orders, and comments. But they never delete them.

A few runs later, your staging database is full of ghost users, orphan orders, and search results polluted with test data.

Or worse, your tests begin failing because they rely on a “clean” state that is no longer clean. Yesterday’s leftovers become today’s flaky test.

Test automation shouldn’t leave a trace. The fix is simple: create data before the test, then clean it up after. Even if the test fails.

import { test as base, expect } from '@playwright/test';

type Product = { id: string; name: string; price: number };

export const test = base.extend<{ testProduct: Product }>({
  testProduct: async ({ request }, use) => {
    const response = await request.post('/api/products', {
      data: { name: 'Test Product', price: 29.99 },
    });

    const product = await response.json();
    await use(product);

    await request.delete(`/api/products/${product.id}`);
  },
});

test('edit product', async ({ page, testProduct }) => {
  await page.goto(`/admin/products/${testProduct.id}`);
  await expect(page.getByText('Test Product')).toBeVisible();
});

That’s the kind of cleanup strategy that keeps test suites reliable and staging data sane.

Does your test suite clean up after itself, or does it leave a mess behind?

Mistake 12/14: You're not catching visual regressions

TestDino — Thu, 16 Apr 2026 13:22:33 +0000

A simple CSS change broke the checkout button on mobile. Every functional test passed.

The scary part? Nobody knew. The button was technically in the DOM. It had the correct text. But to an actual user, it was completely invisible, overlapped by a rogue div on mobile screens only. The kind of bug that gets caught in production when a customer tries to check out and can't.

Functional tests are great for verifying behavior. They are not designed to check appearance. They can't tell you if your app actually looks right.

BEFORE : Functional test passes. Button is covered on mobile.

// ❌ BEFORE — Functional test passes, but the button is covered on mobile.
await expect(page.getByRole('button', { name: 'Checkout' })).toBeVisible();
// Passes because the button is in the DOM, even if users can't actually click it.

AFTER : Visual regression catches layout issues across devices.

// ✅ AFTER — Visual regression catches layout issues across devices.
test('cart page looks correct', async ({ page }) => {
  await page.goto('/cart');

  await expect(page).toHaveScreenshot('cart.png', {
    mask: [
      page.getByTestId('timestamp'),   // changes every run
      page.getByTestId('order-id'),    // unique per session
    ],
    maxDiffPixelRatio: 0.01,
  });
});

// 🧪 Test across viewports in playwright.config.ts
{ name: 'desktop', use: { viewport: { width: 1280, height: 720 } } },
{ name: 'mobile',  use: { viewport: { width: 375, height: 667 } } },

Playwright's built-in visual comparisons (toHaveScreenshot) catch these layout breaks instantly. The quick pro-tip that saves every team time: always mask dynamic elements like timestamps or order IDs to avoid flaky false positives.

The distinction most teams miss is this: functional tests verify that something works. Visual tests verify that users can actually see and interact with it. Both questions matter. Most suites only answer one of them.

When you add visual coverage at the viewport level, you stop shipping layout bugs that your test suite is structurally incapable of catching.

Have visual tests ever caught a bug your functional tests completely missed? Drop it in the comments.

Mistake 11/14: Your E2E tests are checking email format

TestDino — Wed, 15 Apr 2026 13:46:00 +0000

if you are running 400 E2E tests and half of them validate form inputs like email format, you are burning browser time on logic that never needed a browser.

The inconsistency isn't obvious at first. These tests pass. They feel useful. But every one of them navigates to a login page, authenticates, clicks through menus, waits for redirects. All to verify that a regex rejects "not-an-email". That is 30 seconds of browser overhead for a check that has nothing to do with the browser.

BEFORE - E2E for email validation. 30 seconds of overhead.

// Every spec file repeats this flow
test('rejects invalid email', async ({ page }) => {
  await page.goto('/login');                          // navigate
  await page.getByLabel('Email').fill('not-an-email');// fill
  await page.getByRole('button', { name: 'Sign in' }).click();
  await expect(page.getByText('Invalid email')).toBeVisible(); // 30 sec for this
});

AFTER - Component test. Same assertion. 200ms

import { test } from '@playwright/experimental-ct-react';
import { LoginForm } from '../src/components/LoginForm';

test('rejects invalid email', async ({ mount }) => {
  const form = await mount(<LoginForm />);            // no navigation. no login.
  await form.getByLabel('Email').fill('not-an-email');
  await form.getByRole('button', { name: 'Sign in' }).click();
  await expect(form.getByText('Invalid email')).toBeVisible();
});

The distinction most teams miss is this: E2E tests exist to cover what only E2E can cover. Multi-page flows, payment processing, cross-service coordination. That is the territory.

Form validation, input constraints, UI state changes. Those belong at the component level.

When you separate them, everything gets faster and each layer actually earns its place. ⚡

What percentage of your E2E tests are actually testing component behavior? Drop it in the comments.

Playwright Mistake 10/14: Flaky Tests Are Not Random

TestDino — Tue, 14 Apr 2026 13:50:10 +0000

You re-run. It passes. You move on.

Until your whole team is trained to ignore failures. That's when real bugs slip through.

Here are the 3 root causes behind 90% of flaky Playwright tests.

Race Condition

// bad
const text = await page.locator('.welcome').textContent();

// good
await expect(page.getByText('Welcome back')).toBeVisible();
// auto-retries until element exists

Shared Data

// bad
await request.delete('/api/users/test-user-1');
// 5 workers fight over this

// good
const user = await createUniqueUser(request);
await request.delete(`/api/users/${user.id}`);
// each test gets its own

Network Timing

// bad
await page.goto('/search?q=laptop');
await page.locator('.result').first().click();
// results not loaded yet

// good
await page.goto('/search?q=laptop');
await page.waitForResponse('**/api/search');
await page.getByTestId('result').first().click();

Which of these three has burned you the most on your current project?

Mistake 9/14: Stop Relying on Staging Data for Edge Case Testing

TestDino — Mon, 13 Apr 2026 13:04:11 +0000

You finish the happy path tests. Coverage looks solid. Then a user reports the error screen is broken. You never caught it because your suite never actually triggered a server error.

Most teams skip edge cases entirely , or spend hours crafting corrupted data in staging just to trigger one scenario. Then the data gets cleaned up, and the test breaks again.

This is the exact pattern that makes test suites unreliable as they grow.

Playwright's page.route() removes the backend dependency completely. You return whatever response you want : empty arrays, server errors, malformed JSON, slow responses; directly from the test. Your staging environment stays untouched.

The Clean Pattern

Instead of relying on staging state, intercept the network call and return exactly what you need to test:

// Test 1: Empty state
test('shows empty state when no users found', async ({ page }) => {
  await page.route('/api/users', route =>
    route.fulfill({
      status: 200,
      contentType: 'application/json',
      body: JSON.stringify([]),
    })
  );
  await page.goto('/users');
  await expect(page.getByText('No users found')).toBeVisible();
});

Then handle the failure and timeout cases the same way:

// Test 2: Server error
test('shows error message on server failure', async ({ page }) => {
  await page.route('/api/users', route =>
    route.fulfill({
      status: 500,
      body: JSON.stringify({ error: 'Internal Server Error' }),
    })
  );
  await page.goto('/users');
  await expect(page.getByText('Something went wrong')).toBeVisible();
  await expect(page.getByRole('button', { name: 'Retry' })).toBeVisible();
});

// Test 3: Slow response
test('shows loading spinner and timeout message', async ({ page }) => {
  await page.route('/api/users', async route => {
    await new Promise(resolve => setTimeout(resolve, 10000));
    await route.fulfill({ status: 200, body: '[]' });
  });
  await page.goto('/users');
  await expect(page.getByTestId('loading-spinner')).toBeVisible();
  await expect(page.getByText('Request timed out')).toBeVisible({ timeout: 15000 });
});

Zero backend involvement: No staging data setup or cleanup needed
Fully deterministic: The test controls exactly what the API returns
Three edge cases covered: Empty state, server error, and timeout , all in one go

Empty states, error screens, and timeout messages are exactly what users see when things go wrong. Those paths deserve real test coverage, not a "we'll check it manually" comment in the PR.

Do you enforce API mocking for edge cases on your team, or is it still a free-for-all?

Drop it in the comments.

Playwright Mistake 8/14: You Only Check Status Codes on API Responses

TestDino — Sat, 11 Apr 2026 12:36:05 +0000

The docs said the field was a string. Production returned null. Your test passed because you only asserted status === 200.

That's the problem.

What actually happened

The frontend called .toUpperCase() on null. Page crashed. Your test had no idea.

Checking the status code and calling it tested is like confirming the door is closed without checking if it's locked.

These two error messages tell the whole story

"Expected string, received null at 'name'" → 2-minute fix
"Cannot read properties of undefined"      → 2-hour debug

One catches the issue at test time. The other catches it in production.

The fix: validate the response shape

// BEFORE
test('get user profile', async ({ request }) => {
  const response = await request.get('/api/users/123');
  await response.json();
  expect(response.status()).toBe(200);
  // Backend returns { name: null }. Test passes. Frontend breaks.
});

// AFTER
import { z } from 'zod';

const UserSchema = z.object({
  id:        z.string().uuid(),
  name:      z.string(),
  email:     z.string().email(),
  role:      z.enum(['admin', 'member']),
  createdAt: z.string().datetime(),
});

test('get user profile - validate contract', async ({ request }) => {
  const response = await request.get('/api/users/123');
  expect(response.status()).toBe(200);

  const body   = await response.json();
  const result = UserSchema.safeParse(body);

  // name null?       Caught here.
  // field removed?   Caught here.
  // type changed?    Caught here.
  expect(result.success).toBe(true);
});

Why it matters at scale

One method: safeParse covers every field in one call
Visible failure: tells you exactly which field broke and why
No prod surprises: field renames, null returns, type changes all fail at test time

When you have 200+ API tests only checking status codes, you have 200+ tests that can pass while the app is broken.

Tag someone who's still only asserting status codes.

Mistake 7/14: You are still using beforeEach for everything

TestDino — Fri, 10 Apr 2026 13:36:25 +0000

If you are still copy-pasting beforeEachblocks across fifty spec files to initialize your Page Objects, you are building a maintenance trap.

The inconsistency isn't laziness. It is what happens when your setup has no enforced contract across the codebase.

BEFORE - Brittle boilerplate

// Every spec file repeats this logic
let settingsPage;

test.beforeEach(async ({ page }) => {
  settingsPage = new SettingsPage(page);
});

AFTER - Scalable dependency injection

// fixtures.ts
export const test = base.extend({
  settingsPage: async ({ page }, use) => {
    await use(new SettingsPage(page));
  },
});

Playwright built Fixtures to eliminate the "which pattern did we agree on?" conversation.

-Encapsulation: Setup and teardown live together in the fixture.

-Lazy Loading: If a test doesn't ask for the fixture, it never runs.

-Consistency: One source of truth for every Page Object in your framework.

Stop managing state manually. Let the runner do the heavy lifting. ⚡

Do you enforce a fixture strategy on your team, or is it a free-for-all? Drop it in the comments.

Mistake 6/14: Stop Putting Assertions in Page Objects

TestDino — Thu, 09 Apr 2026 13:29:08 +0000

A developer wants to keep their tests "dry," so they put the expect statement inside the Page Object method. It makes sense at first. login() handles everything: fill the form, click the button, confirm you’re on the dashboard. One method. Clean. ✅

Until you need to test a failed login. ✖️

Now your login() method asserts success but your negative test is looking for a failure. To fix it, you add loginExpectingFailure(). Then loginAndCheckError(). Six login methods later, your POM is a graveyard of duplicated logic.

This is the exact trap teams fall into when a suite grows.

Page Objects should be simple. They store locators and perform actions. That is it. Assertions belong in the test file, where the intent of the test is immediately visible.

The Clean Pattern

Instead of baking the assertion into the class, keep the class "pure":

class LoginPage {
  async login(user, pass) {
    await this.emailInput.fill(user);
    await this.passInput.fill(pass);
    await this.submitBtn.click();
  }
}

Then, handle the different outcomes in your actual test files:

// Happy path
test('success', async ({ page, loginPage }) => {
  await loginPage.login('user@test.com', 'pass123');
  await expect(page).toHaveURL('/dashboard');
});

// Negative path
test('failure', async ({ page, loginPage }) => {
  await loginPage.login('user@test.com', 'wrong');
  await expect(loginPage.error).toBeVisible();
});

One method: A single login() method with zero assertions.
Visible Intent: Anyone reading the test knows exactly what is being verified.
No Bloat: No duplicated methods. No boolean flags. No loginVersion4().

Do you enforce a "no assertions in POM" rule on your team, or is it a free-for-all? Drop it in the comments.

playwright #testing #automation #sdet

Mistake 5/14: Your Tests Are Unreadable in the Report

TestDino — Wed, 08 Apr 2026 13:29:19 +0000

A test fails in CI. You open the Playwright HTML report and see 60 lines of code in one giant block. It says "line 47 timed out." You have no idea what the test was actually trying to do. You are left staring at a stack trace guessing which part of the flow broke.

The damage isn't just time. It's clarity. When test results are a pain to decipher, developers stop reading them. Failures get dismissed.

test.step() fixes this.

BEFORE: One giant block. The report just says "X test failed."

test('submit contact form', async ({ page }) => {
  await page.goto('/contact');
  await page.getByLabel('Name').fill('Jane');
  await page.getByLabel('Email').fill('jane@test.com');
  await page.getByRole('button', { name: 'Submit' }).click();
  await expect(page.getByText('Thank you')).toBeVisible();
});

AFTER: Named steps. The report reads like a story.

test('submit contact form', async ({ page }) => {
  await test.step('GIVEN user is on contact page', async () => {
    await page.goto('/contact');
  });

  await test.step('WHEN user fills and submits form', async () => {
    await page.getByLabel('Name').fill('Jane');
    await page.getByLabel('Email').fill('jane@test.com');
    await page.getByRole('button', { name: 'Submit' }).click();
  });

  await test.step('THEN success message appears', async () => {
    await expect(page.getByText('Thank you')).toBeVisible();
  });
});

The rule: Wrap your code in named steps using GIVEN, WHEN, and THEN.

Now your report says "WHEN user fills and submits form failed" instead of "line 47 timed out".

You know exactly what broke instantly. Debugging goes from minutes to seconds.

Do you use test.step() to structure your tests, or is your code one giant block? Drop it in the comments.

Mistake 4/14: Same data breaks Playwright group tests

TestDino — Tue, 07 Apr 2026 13:32:30 +0000

Running tests together? Same "test@test.com" makes workers fight over one user. Quiet fails everywhere. I reviewed a test suite last week where hardcoded data completely killed their pipeline stability. Worker 1 passes, Workers 2-5 throw "email already exists."

BEFORE : Tests fight each other

test('register new user', async ({ page }) => {
  await page.getByLabel('Email').fill('test@test.com'); 
  // Worker 1: passes. Workers 2-5: fail instantly
});

AFTER : The Factory + JSON Combo

Keep your hacks and limits in a static JSON file. Use a Faker factory for everything else. Zero collisions!

import { faker } from '@faker-js/faker';
import invalidInputs from './invalid-inputs.json';

// 1. Factory function for dynamic, unique runs
export function createTestUser() {
  return {
    email: `test-${Date.now()}-${faker.string.alphanumeric(5)}@test.com`,
    password: faker.internet.password({ length: 12 })
  };
}

test('happy path: zero collisions', async ({ page }) => {
  const user = createTestUser();
  await page.getByLabel('Email').fill(user.email);
  await page.getByLabel('Password').fill(user.password);
});

// 2. Static JSON data for deterministic edge cases
test('reject sql injection in name field', async ({ page }) => {
  // invalidInputs.sqlInjection[0] = "' OR 1=1 --"
  await page.getByLabel('Name').fill(invalidInputs.sqlInjection[0]);
  await expect(page.getByText('Invalid input')).toBeVisible(); 
});

Fixed data for hacks. Faker for everything else.

What's your test data strategy? Comments below.