Most SaaS QA tools have a clause buried somewhere in their value proposition: your tests only run here. Whether it's a proprietary scripting language, a closed-source selector engine, or a platform-only runner, the practical effect is the same. If you cancel, your tests stay. You start from scratch.
This isn't an accident. Lock-in through switching costs is easier to build than lock-in through genuine value. Knowing the difference is how engineering teams avoid a decision they'll regret in two years.
The Proprietary Language Trap
Some platforms require you to write tests in their own scripting DSL. It looks like code, but it compiles into something only their runner understands. You can't run it locally. You can't review it in a standard pull request. You can't grep through it the way you would a TypeScript file.
When you cancel, you have a library of files in a format nothing else reads. Your historical test coverage — two years of institutional knowledge about which flows are fragile, which selectors drift, which assertions catch real regressions — is gone. You're not migrating tests. You're rewriting them.
The Record-and-Replay Trap
Others sell no-code test creation via browser recording. Click through your app, export the test, done. This sounds compelling until you try to maintain it six months later.
Recorded tests encode the exact DOM state at the moment of recording: generated CSS class names, positional selectors, brittle XPaths. When your UI changes — and it will — the recording is invalid. Because the test was generated rather than written, there's no logic to update surgically. You re-record from scratch. Every. Time.
What Genuine Ownership Actually Requires
Ownership isn't about where the files sit day-to-day. It's about what you can do with them at any moment — and whether the code itself is yours to keep. Three things have to be true simultaneously.
- Standard, non-proprietary code. Your tests are written in Playwright with TypeScript — not a wrapper, not a fork, not an abstraction layer. If you handed the files to a developer who had never heard of your vendor, they could read and run them without any explanation.
- Instant, unconditional export. A dashboard that stores your tests for performance is fine — faster updates, faster self-healing, faster CI execution — but it should include a single-click export that packages every test as a standard Playwright project: directory structure, config file, spec files, everything. No support ticket. No waiting period. One click.
-
Zero-modification portability. The export runs immediately on GitHub Actions, CircleCI, a Kubernetes job, or a laptop with
npx playwright test. No migration step. No vendor-specific runner. No changes to the files. The tests you built over two years of product development work exactly the same way outside the platform as they do inside it.
Why Centralized Storage Isn't the Same as Lock-In
Keeping your tests in a vendor's database isn't automatically a lock-in strategy — it can be a legitimate architecture decision, as long as the code format stays open.
When a selector breaks because the UI shipped a change, a self-healing system needs to update the test and re-verify it fast. That loop — detect, patch, validate, deploy — is significantly faster when the source of truth is centralized and the vendor controls the full pipeline. A PR-based workflow into your repository adds review cycles and merge queues to a process that needs to be automatic and immediate.
Centralized storage also means your full test history, failure traces, videos, and run analytics live in one place — accessible without any setup on your side. But the underlying code should remain standard Playwright TypeScript throughout. Storage location is an operational detail. Code format is the ownership guarantee.
The Three Questions to Ask Any QA Vendor
- Is the test code written in a standard, open-source format? Proprietary DSLs and no-code recorders create formats only the vendor's platform understands. Standard Playwright TypeScript is readable and runnable anywhere.
- Can I export all my tests right now, without asking? If the answer involves a support request, a data export form, or a waiting period — that's a red flag. Export should be self-serve and immediate.
- What happens to my tests if I cancel today? You should be able to export a complete, runnable Playwright project and continue running it without modification. If the answer is "you'll need to re-create them" — you never owned them.
Your test suite is accumulated institutional knowledge about how your application behaves. It should never be held hostage by a vendor relationship — because a tool you can leave freely is a tool worth trusting.
Originally published on the QA Guardian blog.
Top comments (0)