A checkout test can “heal” and still become wrong.
Imagine that [data-cy="purchase"] disappears. The page now contains two visible buttons labeled Continue: one requests a quote; the other places the order. An AI repair clicks the first match and the test passes because both flows eventually show “Thank you.”
The selector recovered. The purchase behavior is no longer tested.
This is why self-healing needs a change boundary.
I am a Cypress Ambassador. This article complements my CypressConf 2026 session, “When Tests Heal Themselves: Building Trust in AI-Generated Cypress Tests.”
Separate three kinds of change
| Change | Example | Review level |
|---|---|---|
| Selector repair | New role/name for the same action | Focused review |
| Step rewrite | New click, wait, and assertion sequence | Full test review |
| Intent reinterpretation | Different action judged “equivalent” | Product decision |
Calling all three “healing” hides the risk. A tool should be allowed to perform only the class of change the team intended.
A useful policy separates automatic analysis from automatic mutation:
| Stage | Agent may do | Agent may not do |
|---|---|---|
| Diagnose | Read failure evidence and rank causes | Change source files |
| Propose | Produce a bounded patch and evidence summary | Merge or approve its own patch |
| Verify | Run the affected spec and negative control | Redefine the expected behavior |
| Release | Attach review evidence | Bypass branch protections |
This preserves the speed benefit of AI-assisted triage without making “the suite is green” the only approval criterion.
Define the allowed envelope
A simple review policy can state:
const repairPolicy = {
editableFiles: ['cypress/e2e/checkout.cy.ts'],
allowed: ['selector-replacement', 'wait-for-observable-event'],
forbidden: [
'assertion-removal',
'fixed-delay',
'force-click',
'skip-test',
],
requiredPostconditions: [
'POST /api/orders returns 201',
'confirmation route reached',
'persisted order status is confirmed',
],
humanApproval: true,
}
This is an application policy, not a built-in Cypress format. Its purpose is to make “fix the test” reviewable.
Protect intent outside the selector
A robust test uses semantic anchors that survive markup changes:
cy.intercept('POST', '/api/orders').as('createOrder')
cy.findByRole('button', { name: 'Place order' }).click()
cy.wait('@createOrder').then(({ request, response }) => {
expect(request.body.items).to.have.length.greaterThan(0)
expect(response?.statusCode).to.eq(201)
})
cy.location('pathname').should('match', /\/orders\/[^/]+\/confirmation$/)
Now replacing the selector with Request a quote does not satisfy the route and request postconditions.
Useful anchors include:
- accessible role and name;
- destination route;
- API operation and payload;
- durable server state;
- audit event;
- permission check;
- absence of a forbidden side effect.
Choose the smallest combination that uniquely defines the behavior.
Reject suspicious repair patterns
AI-generated diffs deserve extra scrutiny when they add:
cy.wait(5000)
cy.get('button').first().click({ force: true })
it.skip('places an order', () => { /* ... */ })
Other warning signs are removed assertions, broader selectors, increased timeouts, or replacing a domain assertion with visibility alone.
The repair should provide observable evidence:
Failure: old selector no longer matched
Replacement: role=button, name="Place order"
Changed: selector only
Preserved: order request, confirmation route, persisted status
Not changed: assertions, timeout, force options
Treat the source diff and rerun as authoritative. The summary is a review aid, not proof.
Include an environment fingerprint in that evidence: Cypress, Node, browser, application commit, viewport, and CI image or Docker digest. A repair should first reproduce in the original pinned environment. Otherwise an unreviewed browser or image upgrade can be misclassified as a broken selector—and the agent may rewrite a valid test to accommodate runner drift.
Keep the candidate patch narrow enough for a reviewer to understand in one pass. If the proposed fix crosses application source, fixtures, and assertions, split the work: first repair or clarify the product, then update the test against the reviewed behavior. A large cross-layer patch can make a failing product and a weakened assertion cancel each other out.
Run a negative control
After the repaired test passes, introduce a controlled defect that violates the requirement. For example, stub order creation to fail:
cy.intercept('POST', '/api/orders', {
statusCode: 409,
body: { code: 'ORDER_NOT_CREATED' },
})
The purchase test must now fail. If it remains green, the repair weakened the intent.
Negative controls are especially important after step regeneration. They answer the question a successful rerun cannot: Can this test still detect the defect it exists to prevent?
Know the Cypress workflow you are using
cy.prompt() supports generating code that you export and commit, as well as continuous self-healing at runtime. Generated-once code fits ordinary review more naturally because the diff exists before merge. Runtime adaptation needs recorded evidence and tighter restrictions on consequential flows.
Use continuous adaptation cautiously for low-risk presentation changes. Payments, permissions, account deletion, privacy controls, and regulated workflows should normally require an explicit reviewed patch because their business meaning cannot be inferred safely from a nearby passing path.
Cypress currently lists cy.prompt() as Generally Available and Studio AI recommendations as Beta. Release maturity matters, but neither status removes the team's responsibility to define test intent.
The approval rule
Allow automatic repair only when:
- the change is inside the declared envelope;
- independent postconditions preserve the original behavior;
- the code and evidence are reviewable;
- a relevant negative control still fails.
Otherwise, escalate instead of healing.
What is the most consequential behavior in your suite that a “helpful” broad selector could accidentally replace?
Top comments (0)