Many QA teams keep end-to-end automation in a dedicated repo — separate from the app it tests. It's good hygiene: different owners, different release cadence, no test churn polluting app history.
It also quietly removes the one signal that would tell a developer their change broke a test.
The boundary problem
When tests and app share a repo, at least the failing spec shows up in the same CI run and the same blame history. Split them, and the connection vanishes:
- The frontend dev opens a PR in the app repo. CI there is green — there are no E2E tests to run.
- The PR merges. A scheduled or downstream run in the automation repo goes red hours later.
- The QA engineer who owns that repo inherits a failure with no link back to the PR that caused it.
The dev never sees it. The reviewer never sees it. The cost lands on whoever owns the other repo, after the fact. If you're the automation owner, you know this loop: what changed vs what broke vs which specs do I update.
Why normal tooling can't help here
Coverage tools, "affected test" selectors, and CI test-splitting all operate within a single repo and a single test run. They have no concept of "a change here breaks a test there." GitHub's own PR view shows the diff and that repo's checks — nothing about a sibling repo.
The boundary that makes the split clean is the same boundary the tooling can't cross.
What actually connects them: text
Here's the useful observation: your E2E specs are coupled to the app through a small set of textual anchors — data-testid / data-cy attributes, getByRole/getByText targets, aria-labels, element ids, route strings, visible copy.
A PR that renames data-testid="login-btn" or moves /dashboard to /home endangers every spec that references those strings — regardless of which repo the spec lives in. Text doesn't care about repo boundaries. So the check is:
- Extract the anchors a PR's diff touches (added/removed lines only).
- Search the automation repo's spec files for those anchors.
- Anything that matches is a break candidate — surfaced on the PR, at review time.
That's cheap, static, and crosses the boundary that execution-based tooling can't.
Automating it
I'm a QA automation engineer and I built this into a GitHub App — Testward. You add a one-line .testward.yml to the app repo listing your automation repo(s); on every PR it extracts the anchors, scans the linked repos' specs (code search + a tree-grep fallback for large suites), runs an LLM pass to filter candidates that don't actually break, and posts one comment:
🚨 Automation this PR may break:
- acme/web-e2e → tests/login.spec.ts — clicks [data-testid=login-btn]
(renamed to signin-btn) and asserts the /dashboard redirect (changed to /home)
The dev sees the consequence while they still have the context — and either fixes the selector or pings the automation owner before the merge, not after the red.
Being upfront about the limits: the matching is textual, so dynamic selectors (`row-${id}`) won't match — that's the known gap. And if you want to poke at the anchor-extraction idea without installing anything, there's a free in-browser diff scanner that runs entirely client-side.
Takeaway
If your automation lives in its own repo, no amount of in-repo tooling will warn the app team they're about to break it. The dependency is textual, so the check can be too — and it belongs at review time, where the fix costs a comment instead of an afternoon.
If your team runs a separate automation repo, I'd genuinely like to hear how you handle this today — scheduled runs and hope? A sync meeting? Something smarter?
Top comments (0)