How We Built an AI Test Case Reviewer Using Cursor, MCP, and Allure TestOps

Hi! I’m Oleg, a QA Engineering Manager focused on test strategy, automation, and AI-assisted QA workflows.

My goal is simple: eliminate repetitive work for QA engineers using AI tools.

In an ideal world, we provide an AI model with requirements, Figma designs, Git branches, and other project artifacts through MCP. The model then helps us:

1. Create test cases using the available project context.
2. Generate automated tests based on those test cases.

We Decided to Start With AI Test Case Reviews

Writing high-quality test cases from requirements and supporting documentation is not an easy task.

Instead of trying to automate the entire process immediately, we decided to start with something smaller: an AI-powered test case reviewer.

The basic idea was:

1. A QA engineer moves a test case to the Review status in Allure TestOps.
2. An AI agent retrieves and reviews the test case according to our internal guidelines.
3. The agent publishes the review as a new comment on the test case.
4. The test case is moved to an appropriate status, such as Needs Fixing, so the QA engineer immediately knows what to do next.

To implement this workflow, we needed the AI agent to be able to:

• Connect to Allure TestOps.
• Retrieve a test case and its scenario.
• Review the content.
• Publish the review back to TestOps.

For this purpose, we created our own MCP server for Allure TestOps, along with an interface for configuring review prompts for individual TestOps projects.

Why We Did Not Use the Built-in Allure TestOps MCP

Allure TestOps recently introduced its own MCP integration and an AI chat interface directly inside the product.

However, the AI chat requires a separate license. We also discovered the official MCP integration only after we had already developed our own solution.

After examining the built-in TestOps AI functionality, we concluded that our implementation covered our current needs at least as well.

Our solution also remains free for us within our existing infrastructure and software licenses.

How Our MCP Server Works

Our MCP server is written in Python using the FastMCP framework.

We also converted the OpenAPI specification provided by Allure TestOps into a generated API client.

The resulting workflow looks like this:

1. The MCP server calls the TestOps API.
2. It retrieves the required test case data, including its scenario, steps, attachments, and other fields.
3. The data is passed to an LLM for analysis.
4. After processing, the result is saved back to TestOps through MCP—for example, as a new review comment.

Allure TestOps
      ↓
MCP server
      ↓
Language model
      ↓
MCP server
      ↓
TestOps comment

Why We Used Cursor

When we started working on the prototype, we had several practical limitations:

• The web interface for reviewing test cases was not completely ready.
• Our MCP server could occasionally fail to return some parts of a TestOps scenario.
• The language model used by the main solution required separate payment.
• Our team already had access to corporate Cursor accounts and experience using Cursor.

I therefore created a minimal project that could be opened in Cursor.

The goal was to let any QA engineer:

• Clone the repository.
• Open it in Cursor.
• Run a single command to review a test case.
• Quickly evaluate the quality of the AI-generated review.

How the Cursor Project Is Structured

The project contains only a few components:

• One project rule containing the main test case review prompt.
• One command: /review-test-case.
• One user rule that ensures Cursor loads the project rules.
• An MCP server for interacting with Allure TestOps.

User Rule

We added a user rule because Cursor did not always load the project rules consistently.

The rule explicitly instructs Cursor to read the project rules before every response. A fixed phrase at the beginning of the response serves as a visible indication that the rules were actually applied.

Respond in Russian.

Before every response, read the project rules from .cursor/rules.
Do not begin answering until you have read the rules.

Start every response, without exception, with the phrase:
"Rules added to context: X, Y, Z, etc."

List the names of the rule files you have read.

This is a workaround rather than an ideal architecture, but it made the behavior easier to verify during our initial experiments.

The Test Case Review Prompt

The following prompt is still experimental. We created it to obtain initial results quickly and determine which areas required further improvement.

You are a Senior QA Engineer with more than 30 years of
experience testing mobile, web, and API applications.

You review test cases in Allure TestOps and help QA engineers
write cases that are:

- unambiguous;
- verifiable;
- atomic;
- reproducible;
- deterministic;
- suitable for automation.

I will provide screenshots or text from an Allure TestOps test
case. Sometimes only some of its fields will be available.

Your task is to review the test case quickly and strictly
according to the rules below and propose specific corrections.

=== REVIEW GUIDELINES ===

1. General principles

Check the test case for:

- unambiguity;
- verifiability, including measurable expected results;
- atomicity;
- independence and repeatability;
- determinism, including fixed feature flags and test data;
- suitability for automation.

Remove vague phrases such as "verify that it works correctly"
or "pay attention to."

2. Terminology

Use the same terms that appear in the product UI.

If UI terminology differs from API or domain terminology,
explain the difference.

3. Required fields

A test case must include:

- Title;
- Preconditions containing only system state and test data;
- Shared steps if the same sequence appears in two or more cases;
- A scenario with an Expected Result for every step;
- Relevant artifacts;
- Tags and links.

4. Title

4.1 Mobile and frontend tests:

<Screen or section> — <What is being tested> —
<Expected effect>

4.2 API tests in a separate API project:

Suite = <endpoint or context>

Title = <scenario or variation>

Do not include generic terms such as Positive, Negative,
Search, or HTTP in the test case title.

5. Preconditions

Do not include instructions such as:

- Read the documentation.
- Prepare Postman.
- Import the cURL request.

These are actions, not system states.

Specify test data using either exact values or criteria for
finding suitable data. Also explain where to obtain it.

6. Steps

One step must contain one action.

Every Expected Result must be measurable. Specify exactly what
should be displayed, which status code should be returned,
which fields should be present, or which state should change.

=== REQUIRED RESPONSE FORMAT ===

A) Short assessment

✅ What is good

Provide 3–6 points.

❌ Problems and risks

Provide 3–8 points.

Be specific. Examples:

- Expected Result is not measurable.
- Preconditions contain instructions instead of state.
- Test data or selection criteria are missing.
- Several checks are mixed into one test case.
- The scenario is not deterministic.
- Terminology differs from the product.
- Repeated actions should be moved to a shared step.

B) Corrections: before → after

Use the following structure for every correction:

1. What is wrong:
   One sentence explaining why it must be changed.

2. Before:
   A short quotation or summary of the current text.

3. After:
   A concise improved version.

Provide corrections for the following areas:

- Title;
- Preconditions;
- Shared steps, if applicable;
- Scenario, including steps and an Expected Result for every step;
- Artifacts, tags, and links, if applicable.

Important:

- Do not write long theoretical explanations.
- Provide only the review and specific corrections.
- Avoid vague statements such as "the link opens."
- Specify where and how it opens: system browser or WebView,
  expected URL or title, close button, and absence of errors.
- If navigation, token preparation, or environment setup is
  repeated, explicitly recommend moving it to a shared step.
- If an exact value is unknown, use placeholders such as
  {banner_id}, {token}, or {card_number}.
- Explain how to find suitable data and verify that it is current.

=== INPUT DATA ===

The user provides the ID of the test case that must be reviewed.

Start the response immediately with section A, followed by
section B.

The /review-test-case Command

To allow QA engineers to launch a review with a single command, I added the review-test-case command.

# review-test-case

Review a test case in Allure TestOps and publish the result
as a NEW comment on that test case through the TestOps MCP server.

Important:

Always create a new comment.

Never update an existing comment and never delete comments.

Input parameter:

{testcase} — a test case URL or ID.

Data priority:

1. Retrieve the test case data through MCP using
   testcasegetbyid and testcasegetscenario.

2. If some fields in the API response are empty—for example,
   scenario: null or steps: []—but a screenshot is attached
   to the message, use the information visible in the screenshot.

3. Do not state that the scenario or field is empty if its
   contents are visible in the attached image.

Example:

/review-test-case 70756

Response requirements:

- Start with section A, followed by section B.
- Section A must contain:
  - ✅ What is good: 3–6 points.
  - ❌ Problems and risks: 3–8 points.
- Section B must provide "before → after" corrections for:
  - Title;
  - Preconditions;
  - Shared steps, if applicable;
  - Scenario, including an Expected Result for every step;
  - Artifacts, tags, and links, if applicable.

Why We Added a Second MCP Server for Sequential Thinking

We connected two MCP servers to the project:

1. The TestOps MCP server, which retrieves test cases and scenarios and publishes comments.
2. A sequential-thinking MCP server, which encourages the model to analyse the test case step by step.

We expected sequential analysis to be useful for test case reviews in the same way that it can be useful for code-related tasks.

The model can examine the test case in the following order:

Preconditions
    ↓
Steps
    ↓
Expected results
    ↓
Test data
    ↓
Determinism
    ↓
Suitability for automation

This process helps expose common problems:

• Vague or immeasurable expected results.
• Missing test data.
• Several independent checks mixed into one test case.
• Environment or feature-flag dependencies that are not documented.
• Steps that cannot be automated reliably.

This is currently an observation from our experiments rather than proof that the additional MCP server improves every review.

We still need to evaluate its effect using a larger and more representative set of test cases.

What the Workflow Looks Like in Practice

The process is intentionally simple:

1. Open the required test case in Allure TestOps.
2. Run the following command in Cursor:

/review-test-case <test-case-id>

1. Attach a screenshot if necessary.

The screenshot acts as a fallback because the MCP server may occasionally return an incomplete scenario.

1. Cursor retrieves the test case through MCP.
2. The model reviews the test case using the project guidelines.
3. The MCP server publishes the result as a new comment in Allure TestOps.

A few seconds later, the QA engineer can see the review directly in the test case.

What We Learned

This prototype is deliberately small.

It does not attempt to replace an experienced QA engineer or automatically approve test cases. Its purpose is to identify common quality problems and provide specific corrections before a human reviewer examines the case.

Even this minimal implementation can help detect:

• Ambiguous titles.
• Instructions incorrectly placed in preconditions.
• Missing test data.
• Steps without expected results.
• Non-measurable expected results.
• Repeated actions that should become shared steps.
• Scenarios that depend on undocumented state.
• Test cases that are difficult to automate.

The main advantage is that the result appears inside the tool where QA engineers already work.

They do not need to copy the test case into a separate chatbot and then manually transfer the review back.

Conclusion

The complete workflow is straightforward:

QA engineer
    ↓
Cursor command
    ↓
TestOps MCP
    ↓
Test case data
    ↓
LLM review
    ↓
New TestOps comment

For us, this was a practical way to test AI-assisted test case reviews before building a complete web interface.

The next steps are to:

• Improve the review prompt.
• Measure review quality using a larger dataset.
• Reduce incomplete responses from the TestOps API.
• Determine which parts of the workflow can be safely automated.

For a small prototype, I think the result is promising.

If there is enough interest, I will publish a follow-up describing how we evaluate review quality and generate automated API tests from TestOps test cases.

Let’s Work Together

I’m open to QA leadership opportunities and collaborations with teams looking to improve their testing strategy, build scalable test automation, introduce AI-assisted QA workflows, and establish meaningful quality metrics.

If this sounds relevant to your team, feel free to connect with me on LinkedIn.