DEV Community: Matej Štetiar

Playwright Test Data: Seeding a Real Backend for E2E Suites

Matej Štetiar — Thu, 03 Sep 2026 09:44:26 +0000

Playwright test data is the set of database rows or API records your
application needs to already contain before a browser test runs against
it — a logged-in user, their orders, the products those orders
reference — generated deterministically so the same run produces the
same data every time. Unlike unit tests, a Playwright (or Cypress) spec
drives a real browser against a real, running app, which means the
backend behind it needs real rows to serve, not an intercepted network
response. Getting that data right, and getting it there before the
first test starts, is most of what makes a browser E2E suite fast and
non-flaky instead of slow and order-dependent.

Why is E2E test data hard to manage?

Three patterns keep showing up, and each causes a different failure
mode:

Tests create their own data through the UI. A test that needs an order to exist first signs up a user, logs in, adds a product to a cart, and checks out — all before the actual assertion it cares about. That's slow multiplied across every spec that needs similar setup, and it means the thing under test (the UI) is also the thing doing the setup, so a bug in signup breaks fifty unrelated tests.
A shared, mutable test database. If every spec reads and writes the same rows, test order starts to matter: a test that deletes a user breaks a later test that assumed that user still exists. This is one of the most common sources of a suite that passes locally, one file at a time, and fails intermittently in CI when specs run in parallel or in a different order.
Hand-maintained fixture SQL or JSON. A fixtures.sql file or a static users.json works until the schema changes — a column gets renamed, a new required field is added — and the fixture silently stops matching what the app expects, or starts failing inserts with no clear signal about which of forty rows is the problem.

The fix for all three is the same shape: generate the data the suite
needs from a definition (a template), with a fixed seed, right before
the suite runs, and load it in directly rather than through the UI.

How do you manage Playwright test data for E2E suites?

The pattern is: describe the records your app needs as templates, fix a
seed so the same call always produces the same ids and field values,
and generate them as one relational batch so a logged-in test user
actually owns the orders your assertions expect — no separately
generated orders that need their userId patched in afterward.

{
  "seed": 20260903,
  "documents": [
    { "templateId": "tpl_user", "alias": "user", "count": 1,
      "params": { "email": "jane.doe@example.com" } },
    { "templateId": "tpl_order", "alias": "order", "count": 3,
      "relations": { "userId": { "from": "user.id", "strategy": "round-robin" } } }
  ]
}

Posted to the batch generation API, that
returns one user and three orders where every order.userId is that
user's generated id. Because the seed is fixed, the user's name,
email, and order details come back identical on every run, which is
what lets a spec assert something as specific as "Jane Doe has exactly
3 orders" instead of a vaguer "the orders list is non-empty." Small
batches return the records synchronously in the response; a large batch
returns a batchId you poll instead, which matters if a suite seeds
thousands of rows before a load-style E2E run.

This is a different problem from serving mock API responses. If you're
mocking network calls so a frontend can run with no backend at all, see
mock API response data instead — this
post is about the opposite case, where a real backend and database
exist and the browser test exercises them end to end. It's also
different from seeding a human developer's local database for
onboarding, covered in seeding a local dev database;
that data is long-lived and browsed by hand, while E2E test data is
short-lived, created or reset per run or per CI job, and read by
assertions that check exact values.

How do I wire test data generation into Playwright's global setup?

Call the batch API from globalSetup, before any spec file runs, and
save what comes back to a fixture file specs can import:

// playwright.config.ts
export default defineConfig({
  globalSetup: require.resolve('./e2e/global-setup.ts'),
  // ...
});

// e2e/global-setup.ts
import fs from 'node:fs';

export default async function globalSetup() {
  await resetTestDatabase(); // your own truncate/reset code

  const res = await fetch('https://api.jsonfabrica.com/v1/batches', {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${process.env.JSONFABRICA_API_KEY}`,
      'content-type': 'application/json',
    },
    body: JSON.stringify({
      seed: 20260903,
      documents: [
        { templateId: 'tpl_user', alias: 'user', count: 1 },
        {
          templateId: 'tpl_order',
          alias: 'order',
          count: 3,
          relations: { userId: { from: 'user.id', strategy: 'round-robin' } },
        },
      ],
    }),
  });
  const { results } = await res.json();

  // results is a flat array of { batchId, alias, seqNo, templateId,
  // status, result, documentSeed }; group by alias for easy lookup.
  const seeded: Record<string, unknown[]> = {};
  for (const item of results) {
    (seeded[item.alias] ??= []).push(item.result);
  }

  await loadIntoDatabase(seeded); // your own insert code, or POST to your API

  fs.writeFileSync(
    './e2e/fixtures/seeded-data.json',
    JSON.stringify(seeded, null, 2),
  );
}

Specs then import seeded-data.json instead of re-deriving expected
values:

import seeded from '../fixtures/seeded-data.json';

test('order history shows all orders', async ({ page }) => {
  await page.goto(`/users/${seeded.user[0].id}/orders`);
  await expect(page.getByTestId('order-row')).toHaveCount(
    seeded.order.length,
  );
});

Be clear about what's actually happening here: JsonFabrica returns JSON
records over HTTP. It doesn't insert anything into your database and
it isn't a Playwright plugin, fixture, or reporter — resetTestDatabase
and loadIntoDatabase are your own code, the same as they'd be if the
data came from a static fixture file. What changes is where the data
comes from and how it's kept consistent, not who's responsible for
getting it into the app.

The same generated payload can also back an authenticated
storageState: seed the user in globalSetup, log in via an API call
using that user's generated credentials, and save the resulting cookies
so every spec starts already authenticated instead of clicking through
a login form.

Cypress doesn't have a globalSetup hook, but the equivalent is a
before() block in a top-level spec, or a cy.task
that runs Node code outside the browser sandbox — either one making the
same HTTP call to the batch API before the suite's tests run.

How do you isolate E2E test data across parallel workers?

Running specs across multiple Playwright workers against a shared
backend means two workers can both try to draw the "next" sequential
order number or invoice ID at the same time. Namespacing avoids the
collision: pass a sequenceNamespace and variableNamespace on the
batch request, keyed to the worker index (process.env.TEST_PARALLEL_INDEX
in Playwright), so each worker gets its own counters instead of sharing
one. The mechanics of how createSeq/getSeq and namespaces produce
collision-free values are covered in
unique test data generation with sequences and counters —
worth reading if your templates use sequences for order numbers, invoice
numbers, or anything else that has to be unique within a run.

Should E2E tests use a fixed seed or random data?

Both, at different times. A fixed seed is the default: it's what
makes an assertion like "the third order in the list is a $42.00
Widget" stable across a hundred CI runs, and it's required for visual
regression snapshots to stay pixel-stable. But a suite that only ever
runs against the same seed can accumulate hidden assumptions — a test
that happens to pass because generated record #2 always sorts after
record #1, not because the sort logic is actually correct. Periodically
run the suite with the seed dropped (or rotated) to let genuinely
different data flow through and catch order-dependence or
off-by-one assumptions that a fixed seed was quietly hiding.

Whichever mode you're in, schema changes are still a manual step:
adding a required field to the orders table means editing the
tpl_order template by hand, the same way you'd edit a fixture file —
generation doesn't watch your schema for you.

FAQ

How do I seed test data for Playwright tests?
Call a batch generation API from playwright.config.ts's globalSetup,
before any spec runs, and write the created records (with their
generated ids) to a JSON fixture file that your specs import. Reset the
backend to a known empty or baseline state first, then insert the
generated batch, so every run starts from the same data.

Should Playwright tests create their own data through the UI?
Not as the default pattern. Clicking through signup and creation flows
to set up state for every test is slow, and it makes tests
order-dependent when they share a database, which is a common source of
flaky suites. Reserve UI-driven creation for the handful of tests
actually verifying that creation flow, and seed everything else
directly through an API before the suite starts.

What is Playwright global setup used for?
globalSetup is a function Playwright runs once before any test file,
configured via the globalSetup option in playwright.config.ts. It's
the standard place to seed a backend's database, generate an
authenticated storageState, or do any other one-time preparation the
whole suite depends on.

How do you keep parallel E2E test workers from colliding on data?
Give each parallel worker its own namespace when generating data, so
counters and any workflow-scoped state don't overlap between workers
hitting the same backend at once. JsonFabrica's batch API exposes this
as a sequenceNamespace and variableNamespace per request, keyed on
something like the worker index.

Does JsonFabrica integrate directly with Playwright or Cypress?
No. There's no official plugin, fixture, or reporter for either tool.
JsonFabrica is an HTTP API that returns JSON records; you call it from
your own globalSetup, before() hook, or cy.task, and you write the
code that loads the returned records into your application's database
or API.

Should E2E tests use a fixed seed or random data?
Use a fixed seed most of the time, because it makes generated ids and
field values identical across runs, which is what lets an assertion
like "Jane Doe has exactly 3 orders" stay true instead of drifting.
Periodically run the suite with the seed dropped so genuinely random
data can surface assertions that were quietly depending on a specific
generated value instead of the behavior you meant to test.

Deterministic, relational test data is what the batch generation
API is built for — seed it once in your
globalSetup, and every Playwright or Cypress run gets the same known
data to assert against.

AI Agent Test Data Generation via MCP Server

Matej Štetiar — Thu, 03 Sep 2026 09:44:15 +0000

An AI coding agent working inside Claude Desktop or Cursor can read your
code, write new files, and run your test suite — but it can't open a
browser, log into a dashboard, and click "generate" to get a batch of
realistic test data. It has no hands for a UI. AI agent test data
generation only works if there's something the agent can call: a tool
with a defined schema it can invoke mid-session, the same way it calls a
file-write or a shell command. That's exactly what the Model Context
Protocol (MCP) is for, and it's why we shipped
@jsonfabrica/mcp-server
on npm.

What AI agent test data generation requires over MCP

MCP lets an AI client — Claude Desktop, Cursor, or anything else that
speaks the protocol — launch a small local server over stdio and treat
its exposed functions as tools it can call during a conversation. The
agent decides when to call jsonfabrica_generate_from_template the same
way it decides when to call read_file. For that to work, three things
have to exist: a server process the client can start, a set of tool
definitions with typed inputs and outputs, and — underneath all of it —
some actual operation the tool call triggers. MCP server test data
generation is that last piece: the tool call has to result in real,
schema-conformant data coming back, not a stub.

`@jsonfabrica/mcp-server`, concretely

We published @jsonfabrica/mcp-server v0.1.1 as a local MCP server: the
AI client launches it itself over stdio, no separate process to manage,
no port to open. It exposes the JsonFabrica gateway as a set of MCP
tools — jsonfabrica_create_template, jsonfabrica_generate_from_template,
jsonfabrica_generate_adhoc, jsonfabrica_create_batch,
jsonfabrica_create_sequence, and more. Mid-session, an agent can create
a template matching the shape of your User or Order model, generate
a batch of realistic records against it, and drop the result straight
into a fixture file or a seed script — without you leaving the editor to
go configure anything by hand.

Why this required building nothing new

Here's the part worth being explicit about: every one of those MCP
tools is a thin, typed wrapper around an endpoint that already existed
in the JsonFabrica REST API. jsonfabrica_generate_from_template calls
the same generation endpoint a CI pipeline or a seed script would call.
Writing the MCP server was a matter of describing existing requests and
responses as tool schemas — input validation, output shape, a short
description for the model to read — not building new generation logic,
new data models, or a new backend. The API was already the product; the
MCP server just gives it a second front door.

What a UI-first tool would have had to wrap instead

Contrast that with a tool where the primary interface is a dashboard:
form fields, dropdowns, a "generate" button wired to internal state that
was never meant to be called from outside a browser session. Exposing
that to an AI agent means building an API it never had — endpoints,
request validation, auth, versioned responses — essentially rebuilding
the product's backend to have something to wrap. AI agent test data
generation isn't a feature you bolt onto a UI-first product after the
fact; it's a natural consequence of the product being API-first from the
start. If the REST API is solid, wrapping it for MCP is a week of typed
schemas. If it isn't, MCP support means building the API you should have
had all along.

FAQ

How do I connect JsonFabrica to Claude Desktop or Cursor?
Install @jsonfabrica/mcp-server from npm and add it as an MCP server in
your client's config. The client launches the server itself over stdio, so
there's no separate process to run or port to open, and the agent can then
call its tools directly in a session.

What is an MCP server, and why does it matter for AI coding agents?
MCP, the Model Context Protocol, lets an AI client like Claude Desktop or
Cursor launch a small local server and treat its exposed functions as
tools it can call mid-conversation, the same way it calls a file-write or
a shell command. Without it, an agent has no way to invoke an external
service like a test data API, since it can't open a browser and click
through a UI.

Can an AI agent generate relational or batch test data through MCP, not just single records?
Yes — jsonfabrica_create_batch wraps the same batch generation endpoint
the REST API and CI pipelines use, so an agent can generate a customer and
a set of linked orders in one call during a coding session, not just
isolated single documents.

Does JsonFabrica's MCP server require a separate backend from the REST API?
No. Every MCP tool, such as jsonfabrica_generate_from_template or
jsonfabrica_create_sequence, is a thin typed wrapper around an endpoint
that already exists in the JsonFabrica REST API — there's no separate
generation logic or data model behind the MCP server.

What this actually changes in a coding session

In practice, it collapses a context switch. Instead of stopping to write
a one-off fixture by hand, or tabbing to a dashboard to generate a CSV
and importing it back, an agent working on a PR can generate the test
data it needs — realistic, schema-conformant, matching the model it's
currently editing — as part of the same conversation that's writing the
tests. No manual step, no separate tool, no copy-pasting JSON between
windows. That's the practical payoff of MCP server test data generation:
not a new capability bolted onto the model, but an existing capability
finally reachable from where the work is actually happening.

JsonFabrica vs. Mockaroo vs. Faker.js for Test Data Generation

Matej Štetiar — Thu, 03 Sep 2026 09:43:56 +0000

If you're generating test data today, you've probably landed on one of
three approaches: click through a UI like Mockaroo, pull in a library
like Faker.js and write generation code yourself, or call a hosted API
like JsonFabrica. Comparing these test data generation tools side by
side, the real differences aren't about which one produces "better"
fake data — Faker.js, Mockaroo, and JsonFabrica are all capable of
that. The differences are about where the tool lives, how it handles
relationships between records, and who's responsible for running it.

Three test data generation tools compared, shape by shape

Mockaroo is a browser-based UI: you define columns and types through a
web form, preview rows, and export a file — or hit its API directly,
which is available even on the free tier (paid tiers raise the volume
ceiling rather than gate API access itself). Faker.js is a JavaScript
library: you import it into your own code and call functions like
faker.person.fullName() or
faker.internet.email() to build up objects yourself, one field at a
time. JsonFabrica is an API-first hosted service: you send a schema (or
use a template) to an endpoint and get structured, schema-conformant
JSON back, with no UI step and no library to install in your own
codebase.

That distinction matters more than it sounds. A UI tool is something a
person operates by hand. A library is something a developer owns and
maintains inside their own project — you write the loops, the
relationships, the edge cases. An API-first tool is infrastructure:
something your CI pipeline, your seed script, or an AI coding agent can
call directly, without a human in the loop or generation logic living
in your repo.

UI vs. library vs. API, in practice

Mockaroo's UI is genuinely fast for a one-off task — sketch a schema,
click generate, download a CSV or JSON file. What it isn't built for is
wiring generation into an automated pipeline where nobody is clicking
anything. Its API can cover that, but at free-tier volumes (200
requests/day, up to 1,000 rows per call) it's still built around the
UI-first workflow — for heavier automated use, you either pay for more
API volume or reach for a Mockaroo alternative built API-first from the
start.

Faker.js sits at the opposite end. It's a library, not a service, so
there's no hosting, no account, and no network call — you generate data
in-process, in whatever language your project already uses (there are
Faker ports for several languages — the original Faker was a Perl
library, later ported to Ruby, PHP, Python, Java, and others; Faker.js
is itself one of those ports, not the original). The tradeoff is that
Faker.js gives you field generators, not a data model. If you want an order that references a
real customer ID, or a set of line items that sum to the order total,
you write that logic yourself, by hand, in every project that needs it.

JsonFabrica's tradeoff is the mirror image: you don't write generation
logic, because the schema (or a prebuilt template) describes what you
want and the service produces it, relationships and all. What you give
up is the "no dependency" simplicity of a library — you're calling a
hosted API instead of importing a package.

Schema and relational data support

This is where the three tools diverge most. Faker.js has no concept of
a schema or a relationship between generated objects — it's a toolbox
of individual field generators, and any structure across records
(foreign keys, consistent totals, matching timestamps) is code you
write and maintain. Mockaroo lets you define a schema per dataset and
has some support for relating fields within a single generation, but
it's still fundamentally a spreadsheet-shaped tool: one flat schema,
one export, one dataset at a time.

JsonFabrica is schema-driven and template-based generation by design,
not an add-on. You describe the shape of a record — or reference a
template — and the engine fills it in against real-world patterns
rather than pure randomness, so an order's shipped_at comes after its
created_at, and a discount_code points at a code that actually
exists. Referential integrity across related records — customers that
own orders, orders that own line items — is handled by the generation
engine itself, not left to your seed script to stitch together after
the fact.

JsonFabrica also has stateful sequence functions (createSeq /
getSeq) for cases where you need collision-free unique IDs or
incrementing values across separate test runs — something neither a
stateless library like Faker.js nor a per-export UI tool like Mockaroo
is built to track between calls.

Hosted vs. self-run

Faker.js runs entirely inside your own process — no network dependency,
no data leaving your machine, no bill. That's a real advantage if
you want full control and zero external dependencies for a CI job. The
cost is that you own everything: version upgrades, keeping generation
logic consistent across projects, and building relational logic
yourself if you need it.

Mockaroo is hosted, but its primary interface is the UI; its API is
available free, and paid tiers raise how much you can pull through it
rather than unlocking API access itself. JsonFabrica is hosted and
API-first from the ground up — every feature the UI could offer is available as
an HTTP call, described in a published OpenAPI spec, so it fits into a
CI pipeline or seed script the same way any other API dependency does.
JsonFabrica also ships an MCP server, so AI coding agents in tools like
Claude Desktop or Cursor can request schema-conformant test data
directly in a session, without a person switching to a browser tab.

The tradeoff for going hosted is the one you'd expect: you depend on an
external service being up, and usage is metered — JsonFabrica bills on
a credit-based model rather than a flat license, so cost scales with
how much data you actually generate.

Picking between them

If you need a quick, one-off dataset and don't mind a manual export, a
UI-based tool like Mockaroo is fast and low-friction. If you want
individual fake field values inside code you already control, and
you're fine writing your own relational logic, Faker.js is a solid,
zero-dependency Faker.js alternative — arguably the default choice for
in-process unit tests. If you need schema-conformant, relationally
consistent test data as part of an automated pipeline, seed script, or
AI agent workflow — without hand-writing the logic that keeps records
consistent with each other — that's the gap JsonFabrica is built to
fill: an API-first, schema-driven service with referential integrity
and sequence functions built in, rather than bolted on after the fact.

FAQ

Is Mockaroo free to use?
Yes — Mockaroo's free tier covers the UI and direct API access, capped at
200 requests per day and up to 1,000 rows per call. Paid tiers raise those
volume limits rather than unlocking API access itself.

Is Faker.js the original Faker library?
No. The original Faker was a Perl library, later ported to Ruby, PHP,
Python, Java, and other languages. Faker.js is the JavaScript port of that
same lineage, not the original implementation.

Can Faker.js generate related records, like an order linked to a real customer ID?
Not by itself. Faker.js is a toolbox of individual field generators with no
concept of a schema or a relationship between generated objects — wiring a
foreign key or keeping totals consistent across records is code you write
and maintain yourself, which is exactly the gap schema-driven tools like
JsonFabrica close.

Does JsonFabrica have a free plan or trial?
JsonFabrica's Starter plan includes a 30-day free trial. A
valid payment method is required at signup, but you're not charged until
the trial converts; Growth and Business plans bill immediately with no
trial.

None of these tools are strictly better than the others across the
board. They're built for different points in the workflow, and the
right choice depends on whether you're clicking through a one-off
export, writing generation code by hand, or wiring test data into a
pipeline that has to run without anyone watching it.

Why API-First Wins for Test Data Generation

Matej Štetiar — Thu, 03 Sep 2026 09:43:38 +0000

Plenty of test data tools are built as a UI first and an API second, if
there's an API at all. You open a dashboard, configure some fields,
click "generate," and download a file. That works fine for a one-off
demo. It falls apart the moment test data generation needs to be part of
your actual engineering workflow — running in CI, seeding a database on
every branch, or producing ten thousand records instead of ten. That's
the case for a test data generation API over a click-driven dashboard:
the primary interface is a request you can make from code, and
everything else — a UI, a CLI — is built on top of that same API.

Automation and CI integration

A UI is something a person operates. CI doesn't have a person sitting at
it. If test data generation only exists behind a login screen and a
click, it can't run as a step in your pipeline — someone has to
generate the data ahead of time, commit it, and hope it doesn't drift
from what the tests actually need. An API-first tool is just another
HTTP call your pipeline makes: fetch fresh, schema-conformant data as
part of the build, every run, with no manual step in between.

Scriptability — no clicking required

Generating test data through a UI means clicking through the same
sequence of dropdowns and fields every time you need a new batch. That's
tedious for one dataset and untenable for the dozens of shapes a real
test suite needs — different entity types, different edge cases,
different volumes. An API call is a script. Write it once, parametrize
it, and reuse it for every collection you need, without a human
repeating the same clicks.

Wiring a test data generation API into pipelines and seed scripts

Seed scripts are code that runs at a specific point in a workflow —
before a test suite, on container startup, in a migration. They need a
function call or an HTTP request they can invoke programmatically, not
a browser tab. With a test data generation API, "seed the dev database
with realistic orders" is a line in a setup script, not a manual task
someone has to remember to do before each demo or test run.

Language and framework agnostic

A UI locks you into whatever the tool's frontend supports. An HTTP API
doesn't care what you're writing — a Python test suite, a Go seed
script, a Node.js CI job, or a shell script with curl, can all call
the same endpoint the same way. JsonFabrica is built this way
deliberately: the API is the product, so it works identically whether
you're calling it from a Jest test, a Django management command, or a
Makefile target. You're not choosing a data tool that happens to support
your stack; you're using a data tool that has no opinion about your
stack at all.

Versionable, reviewable requests

A UI configuration lives in a database somewhere, edited by clicking
through screens, with no diff and no history beyond an audit log if
you're lucky. An API request is a payload — JSON, typically — that you
can commit to a repo, put next to your test suite, and review in a pull
request like any other code change. When someone adjusts the shape of
generated test data, that change shows up as a diff, gets reviewed, and
has a commit message explaining why. Config-as-code isn't a nice-to-have
here, it's what makes test data changes auditable instead of invisible.

Scaling to bulk and synthetic data needs

Clicking "generate" in a UI is fine for ten records. It is not how
anyone produces the ten thousand records needed to load-test a database
or populate a staging environment with realistic volume. An API call
takes a count parameter. Generating 50,000 records programmatically is
the same amount of effort as generating 50 — one request, one number
changed. That gap is exactly where UI-only and template-only tools
without an API stop being useful, and it's exactly where an API-first
tool keeps working.

FAQ

What is a test data generation API, and how is it different from a UI tool?
A test data generation API is an HTTP endpoint your code calls to get
structured, schema-conformant data back, instead of a dashboard a person
clicks through. That makes it callable from CI pipelines, seed scripts, and
AI agents without a human in the loop, which a UI-only tool can't support.

Can I call a test data generation API with curl or from any language?
Yes — it's a plain HTTP endpoint, so anything that can make an HTTP
request, including curl, a Python test suite, a Go seed script, or a
Node.js CI job, can call it the same way, with no client library or
dashboard required.

How do I generate bulk test data instead of a handful of records?
Pass a count parameter on the generation request. Generating 50,000
records programmatically is the same amount of effort as generating 50 —
one request, one number changed, instead of clicking "generate" repeatedly
in a UI.

Can I run test data generation as part of a CI pipeline?
Yes — that's the core use case for an API-first tool. Because the
generation call is just an HTTP request, it can run as a step in a CI
pipeline or a container startup script, fetching fresh, schema-conformant
data on every run with no manual export step.

The takeaway

A UI is a convenience layer, and a good one is worth having. But it
can't be the foundation, because automation, CI, seed scripts, and bulk
generation all need something a browser click can't provide: a
programmatic, scriptable, versionable interface. Build the API first and
the UI can sit on top of it. Build the UI first and the API — if it
ever arrives — is usually an afterthought that doesn't cover everything
the UI does. JsonFabrica is built API-first for exactly this reason:
structured, schema-conformant data, generated by a request you can
script, version, and run anywhere.