DEV Community

AI and QA Leaders profile image Let's Automate 🛡️
Let's Automate 🛡️ for AI and QA Leaders

Posted on • Originally published at Medium on

AI-Powered Cypress Test Generation from Natural Language v2.0 — Now with cy.prompt() Self-Healing

#openai #ai #softwaretesting #cypress

AI-Powered Cypress Test Generation from Natural Language — Now with cy.prompt() Self-Healing

Transform plain English requirements into production-ready Cypress tests using GPT-4, LangChain, and LangGraph — run locally or in CI/CD

My Open-source project: github.com/aiqualitylab/cypress-natural-language-tests, which utilizes Cypress’s official AI-powered cy.prompt() command introduced at CypressConf 2025.


AI-Powered Cypress Test Generation from Natural Language v2.0 — Now with cy.prompt() Self-Healing

Introduction

Testing shouldn’t be complicated. You know what your application should do — why spend hours writing boilerplate test code?

I built cypress-natural-language-tests to bridge the gap between your test ideas and working Cypress code. Just describe your test in plain English:

python qa_automation.py "Test user login with valid credentials" --run
Enter fullscreen mode Exit fullscreen mode

Result: A complete .cy.js file generated and executed automatically!

And now, with the latest update, the framework also supports Cypress’s new cy.prompt() command for self-healing, AI-powered test execution.

What’s New: cy.prompt() Integration

Cypress recently launched cy.prompt() — their official AI command that converts natural language into test steps at runtime. My framework now supports both approaches:

Mode Description Best For Generate Mode Creates complete .cy.js test files Version control, CI/CD pipelines cy.prompt() Mode Generates tests using cy.prompt() syntax Self-healing tests, rapid prototyping

You choose what works best for your workflow!

How It Works

👆 The complete workflow — from requirements to executed tests

The framework supports two execution paths :

🖥️ Local Machine Flow v/s ⚙️ GitHub Actions CI Flow


🖥️ Local Machine Flow v/s ⚙️ GitHub Actions CI Flow

Two Powerful Modes

Mode 1: Traditional Test Generation

Generate standard Cypress test files that you own and version control:

python qa_automation.py "Test user login with valid credentials"
Enter fullscreen mode Exit fullscreen mode

Output: 01_test-user-login_20241223_102030.cy.js

describe('User Login', () => {
  it('should login successfully with valid credentials', () => {
    cy.visit('https://the-internet.herokuapp.com/login');
    cy.get('#username').type('tomsmith');
    cy.get('#password').type('SuperSecretPassword!');
    cy.get('button[type="submit"]').click();
    cy.get('.flash.success').should('be.visible');
  });
});
Enter fullscreen mode Exit fullscreen mode

Mode 2: cy.prompt() Generation

Generate tests using Cypress’s new AI-powered cy.prompt() command for self-healing capabilities:

python qa_automation.py "Test user login" --use-cyprompt
Enter fullscreen mode Exit fullscreen mode

Output: 01_test-user-login_20241223_102030.cy.js

describe('User Login', () => {
  it('should login successfully with valid credentials', () => {
    cy.prompt([
      'Visit the login page at https://the-internet.herokuapp.com/login',
      'Type "tomsmith" in the username field',
      'Type "SuperSecretPassword!" in the password field',
      'Click the login button',
      'Verify the success message is visible'
    ]);
  });
});
Enter fullscreen mode Exit fullscreen mode

Why cy.prompt()?

🔄 Self-healing : Tests adapt when UI changes

📝 Readable : Natural language steps in your test files

🛡️ Resilient : Less maintenance when selectors change

Quick Start

Installation 

# Clone the repository
git clone https://github.com/aiqualitylab/cypress-natural-language-tests.git
cd cypress-natural-language-tests

# Set up Python environment
python -m venv venv
source venv/bin/activate # Windows: venv\Scripts\activate
pip install -r requirements.txt

# Configure OpenAI API key
echo "OPENAI_API_KEY=your_key_here" > .env

# Initialize Cypress
npm install cypress --save-dev
npx cypress open
Enter fullscreen mode Exit fullscreen mode

Generate Your First Test 

# Standard Cypress test
python qa_automation.py "Test user registration flow"

# With cy.prompt() syntax
python qa_automation.py "Test user registration flow" --use -cyprompt

# Generate and run immediately
python qa_automation.py "Test homepage loads correctly" --run
Enter fullscreen mode Exit fullscreen mode

Practical Examples

Example 1: Multiple Test Requirements 

python qa_automation.py \
  "Test successful login with valid credentials" \
  "Test login fails with wrong password" \
  "Test login form shows validation errors for empty fields"
Enter fullscreen mode Exit fullscreen mode

Creates three separate test files — one for each requirement.

Example 2: With Documentation Context (RAG)

Supercharge test generation with your own documentation:

python qa_automation.py \
  "Test checkout API according to specifications" \
  --docs ./api-documentation \
  --persist-vstore
Enter fullscreen mode Exit fullscreen mode

The framework indexes your docs into ChromaDB and uses them as context for more accurate test generation.

Example 3: Generate and Execute Locally 

python qa_automation.py "Test user profile update" --run
Enter fullscreen mode Exit fullscreen mode

Generates the test AND runs Cypress immediately. View results in your terminal.

Example 4: CI/CD Integration

Trigger via GitHub Actions to generate tests in your pipeline:

- name: Generate Tests
  run: python qa_automation.py "${{ github.event.inputs.requirement }}"

- name: Run Cypress
  run: npx cypress run

- name: Upload Artifacts
  uses: actions/upload-artifact@v3
  with:
    name: cypress-results
    path: |
      cypress/videos
      cypress/screenshots
Enter fullscreen mode Exit fullscreen mode

Why Choose This Framework?

Feature Benefit Dual Mode Support Standard Cypress OR cy.prompt() — your choice Complete Test Files Version control your generated tests Documentation-Aware RAG integration for accurate, context-rich tests Local & CI Ready Works on your machine and in GitHub Actions Model Flexibility Use GPT-4, GPT-4o-mini, or GPT-3.5-turbo Open Source Full control, no vendor lock-in

Configuration

Change AI Model

In qa_automation.py:

llm = ChatOpenAI(
    model="gpt-4o-mini", # Options: gpt-4, gpt-4o, gpt-3.5-turbo
    temperature=0
)
Enter fullscreen mode Exit fullscreen mode

Set Your Application URL

Update the prompt template to target your application:

CY_PROMPT_TEMPLATE = """
...
- Use `cy.visit('https://your-app-url.com')` as the base URL.
...
"""
Enter fullscreen mode Exit fullscreen mode

Get Started Now

🔗 github.com/aiqualitylab/cypress-natural-language-tests

git clone https://github.com/aiqualitylab/cypress-natural-language-tests.git
Enter fullscreen mode Exit fullscreen mode

⭐ Star the repo if you find it useful!

Conclusion

Natural language test generation is here to stay. With cypress-natural-language-tests , you get:

Two modes  — Traditional Cypress or cy.prompt()

Full ownership  — Complete test files you control

CI/CD ready  — Works locally and in GitHub Actions

Documentation-aware  — RAG for accurate test generation

Open source  — No vendor lock-in

Stop writing boilerplate. Start describing tests in plain English.

What’s your experience with AI-powered test generation? Drop a comment below!

Top comments (0)