DEV Community: CloudQA

How to Prompt an LLM for Test Cases You Can Actually Use

CloudQA — Wed, 14 Oct 2026 04:32:19 +0000

Originally published on the CloudQA blog.

Ask an LLM to "write test cases for a login page" and you'll get something that looks impressive and tests almost nothing. The output is fluent, well formatted, and generic: valid login, invalid password, empty fields. Any junior tester could have listed those in two minutes.

The model isn't the problem. The prompt is. An LLM can only work with what you give it, and "write test cases for login" gives it nothing about your product, your rules, or what counts as a failure.

This guide covers a simple prompt structure, five prompts you can adapt today, and the review habits that keep AI-generated tests from becoming a liability.

The four parts of a useful test-generation prompt

Whichever model you use, prompts that produce usable test cases tend to contain the same four things:

A role. "Act as a senior QA engineer" sets the level and vocabulary of the response. It's a small lever, but it works.
Real context. Paste the actual user story, acceptance criteria, API spec, or a code snippet. This matters more than anything else on the list. A model reading your real requirements can find real gaps. A model reading "a login form" can only guess.
A specific task. Use direct verbs: "Generate," "List," "Identify." Avoid "think about" or "consider."
An output format. Ask for a table with named columns, Gherkin, or JSON. If you want to paste the result into a test management tool, say so.

If you only fix one thing, fix the context.

Prompt 1: Functional test cases from a user story

The most common use case. You have a story and acceptance criteria and need a first pass at coverage.

Act as a senior QA engineer. Based on the user story and acceptance
criteria below, generate functional test cases as a table with the
columns: Test Case ID, Scenario, Preconditions, Steps, Expected Result.

Include positive, negative, and boundary cases. If any acceptance
criterion is ambiguous, list your assumption in a separate "Assumptions"
section instead of silently choosing an interpretation.

User story:
As a registered user, I want to add a product to my cart from the
product details page so that I can purchase it later.

Acceptance criteria:
1. The product details page displays an "Add to Cart" button.
2. Clicking it adds the selected item to the cart.
3. A confirmation message appears: "Product added to cart successfully."
4. The cart icon in the header updates to show the new item count.
5. If the item is already in the cart, the button stays clickable and
   the cart quantity increases.

The "Assumptions" instruction is the most valuable line in that prompt. It makes the model show its reasoning about gaps in your requirements instead of papering over them, and those gaps are often the real bugs.

Here's what good output looks like (an illustrative example, written by hand):

ID	Scenario	Steps	Expected Result
TC-01	Add a new item	Open product page, click Add to Cart	Item appears in cart, confirmation shown, header count +1
TC-02	Add an item already in cart	Add the same product twice	Quantity becomes 2, header count updates correctly
TC-03	Add while out of stock	Open an out-of-stock product	Button disabled or clear error shown, cart unchanged
TC-04	Rapid repeated clicks	Click Add to Cart five times quickly	Quantity is consistent, no duplicate line items

TC-03 and TC-04 aren't in the acceptance criteria. Cases like these are where the model earns its keep, and where you should look first when reviewing.

Prompt 2: Edge cases and negative scenarios

Models are good at brainstorming the unusual inputs people forget.

Act as a meticulous QA tester. For the login form described below,
list negative scenarios and edge cases. Focus on invalid input,
boundary conditions, and unusual user behavior. Group them by
category and, for each one, state what you expect the system to do.

Functionality:
A login form with "Email" (must be a valid email format) and
"Password" (minimum 8 characters).

Ask for categories (input validation, session behavior, rate limiting, accessibility). Unstructured lists drift toward the obvious cases.

Prompt 3: Security-focused test ideas

For testing your own applications, a model can quickly generate a checklist of common vulnerability checks. Only run these against systems you own or are explicitly authorized to test.

Act as an application security tester. For a registration form that
accepts username, email, and password, generate test cases to check
for common input-handling vulnerabilities such as SQL injection and
cross-site scripting. For each case, give the input, the field it
targets, and the secure behavior you'd expect.

Treat the output as a starting checklist, not a security assessment. It doesn't replace a proper review or dedicated security tooling.

Prompt 4: BDD scenarios in Gherkin

If your team writes behavior-driven specs, you can generate a first draft of the feature file.

Act as a BDD specialist. Write a Gherkin feature file for the feature
below, with multiple Scenario blocks covering positive and negative
paths. Use Scenario Outline with Examples where inputs vary.

Feature: Product search
As a user, I want to search for products by name so that I can find
what I'm looking for.

A trimmed example of the kind of output to expect:

Scenario: Search with a matching product name
  Given I am on the home page
  When I search for "wireless mouse"
  Then I should see products whose names contain "wireless mouse"

Scenario: Search with no matches
  Given I am on the home page
  When I search for "zzzzqqq"
  Then I should see a "no results found" message

Prompt 5: Ask the model to critique its own work

This one is underused. After generating a suite, run a second prompt against it:

Review the test cases below against the requirements I provided.
1. Which requirements have no test coverage?
2. Which test cases are duplicates or near-duplicates?
3. Which test cases assume behavior that isn't stated in the
   requirements?
4. What are the five highest-risk scenarios that are missing?

Question 3 is the important one. It surfaces cases where the model invented behavior, which is the most common failure in AI-generated tests.

Where AI-generated tests go wrong

LLMs are good first-draft engines. They are not oracles.

They don't know your business. A model doesn't know that refunds over a certain amount need manager approval, or that one customer tier gets special pricing. It only knows what's in your prompt.
They invent plausible behavior. If the requirements are silent, the model may fill in an answer that sounds right and is wrong for your product. Tests built on invented behavior fail for the wrong reasons, or worse, pass for the wrong reasons.
They produce volume, not priority. Thirty test cases isn't better than ten if you can't tell which five matter.
Generated automation code is a draft. If you ask for Playwright or WebdriverIO scripts, check every selector and wait strategy against your real application. Generated selectors are often guesses.
Data privacy is your responsibility. Don't paste proprietary code, customer data, or confidential requirements into a public model interface. Use an enterprise offering with the data guarantees your organization requires.

A short review checklist

Before any AI-generated test case enters your suite:

Does it trace back to a real requirement?
Are the expected results stated explicitly, or just implied?
Is any behavior assumed that you haven't confirmed?
Is it a duplicate of something you already have?
Would you know why it failed if it failed?
Is it worth automating, or is it better as an exploratory check?

If a test case can't pass that list, rewrite it or drop it.

From prompt to automation

The usual workflow is: generate cases, review them, pick the ones worth automating, then turn those into scripts. You can ask a model for framework-specific code:

Convert test case TC-02 into a Playwright test in TypeScript. Use
role-based locators where possible, avoid fixed timeouts, and add a
comment above any selector you're unsure about.

That last instruction, flagging uncertain selectors, gives you a built-in list of things to verify by hand.

If you'd rather not hand-write prompts at all, CloudQA's AI test case generator produces test cases from a feature description in the browser.

The takeaway

Better prompts come from better inputs: real requirements, an explicit format, and a request for the model to state its assumptions. The model drafts quickly. Your job is to judge what's worth keeping.

CloudQA builds codeless test automation for web applications. If you're working on test case generation or reducing test maintenance, we'd be glad to hear what you're working on.

End-to-End Salesforce Automation: Lightning Components, a Dynamic DOM, and OTP MFA

CloudQA — Tue, 06 Oct 2026 17:24:29 +0000

Originally published on the CloudQA blog.

Salesforce is a genuinely difficult platform to automate well, and the difficulty is structural rather than incidental. Three characteristics of the platform combine to make naive automation approaches fail quickly.

The Problem

A dynamic DOM. Salesforce's Lightning interface generates much of its markup dynamically, and element attributes — including IDs and classes automation would normally rely on — can change between sessions or even between renders of the same page. Selectors that work today can silently stop working tomorrow, with no code change on the testing side to explain why.

Lightning components. Salesforce's component framework renders complex UI elements — dynamic forms, dropdowns, data tables, pop-ups — through its own internal structure rather than plain HTML controls. These components often don't expose the kind of simple, stable attributes that standard selector strategies expect, which makes reliably identifying and interacting with them harder than automating a conventional web form.

OTP-based MFA. Salesforce's one-time-password multi-factor authentication is, like most MFA, deliberately resistant to automated bypass. A test suite that has to clear an OTP challenge on every run either can't run unattended at all, or has to solve session persistence some other way.

Compounding all three: the goal wasn't to verify isolated pages in a vacuum, but to automate complete business workflows — multi-step processes spanning several Salesforce screens, involving navigation, data entry, validation, and record updates in sequence. Any one unstable step in a long workflow can break the entire run, which raises the stakes on getting each of these problems solved properly rather than patched around.

What We Built

Stable selector strategy for a dynamic DOM

Rather than relying on the kind of single-attribute selectors that break the moment Salesforce regenerates its markup, we identified genuinely stable selectors where they existed and built custom selector strategies for the elements where they didn't. That meant looking past whatever attribute seemed most convenient at first glance, and instead identifying the properties of each element that actually stayed consistent across sessions and re-renders — the difference between automation that survives Salesforce's normal behavior and automation that has to be rewritten every time the DOM regenerates.

Solving OTP/MFA with a persistent authenticated session

As with session-based authentication generally, trying to script around an OTP challenge on every execution is both fragile and, for most implementations, not a reliable long-term approach. Instead, we used a dedicated Chrome Debugger Profile to maintain an already-authenticated Salesforce session. Because the profile retains valid login state between runs, automated executions connect to a session that has already cleared OTP validation, rather than triggering a fresh MFA challenge every time.

A dedicated, isolated execution environment

We configured a dedicated VM specifically for Salesforce test execution, separate from other automation infrastructure. This gives Salesforce automation a consistent environment run after run — no state bleed from unrelated projects, no resource contention, and a much shorter path to diagnosing a failure when one does occur, since the environment itself is a known, controlled variable rather than a moving target.

Automating Salesforce's complex UI components

Salesforce-specific components — dynamic forms, dropdowns, data tables, pop-ups, and Lightning components generally — needed automation logic built for how they actually behave, not treated like standard HTML controls. That meant building interaction strategies specific to each component type: handling dropdowns that render their options dynamically, tables that load and paginate data asynchronously, and pop-ups that need to be detected and handled without stalling the workflow behind them.

Selector maintenance and healing

To keep the suite stable as Salesforce elements and attributes change over time, we used fallback selector strategies so that when a primary selector no longer matches, the automation can still locate the correct element through an alternate identifying property. Just as important: when a selector does need to be updated, it can be updated directly — without re-recording the entire test from scratch. That distinction matters over the life of a test suite; the cost of Salesforce's inevitable UI drift becomes a small, targeted fix instead of a rebuild.

Full end-to-end workflow automation

With the underlying stability problems solved, we automated complete business workflows across multiple Salesforce screens — not individual pages checked in isolation, but full processes: navigating between screens, entering data, validating it, and updating records, all within a single automated run that mirrors how a real user actually moves through the system. This is a meaningfully higher bar than page-level testing, since a failure anywhere in the chain has to be caught, and every step depends on the ones before it having completed correctly.

The Outcome

The result is a Salesforce automation suite that runs complete, multi-screen business workflows unattended — without manual OTP entry, without breaking every time Salesforce regenerates an attribute, and without needing a full test rewrite every time a Lightning component's markup shifts. By solving authentication, environment stability, and selector resilience as foundational problems first, the workflows built on top of them — the actual business processes the project needed covered — could be automated end-to-end rather than only in fragile pieces.

Key Takeaways

Treat Salesforce's dynamic DOM as the default, not the exception. Selector strategy has to account for change from day one, not react to it after tests start failing.
Persistent authenticated sessions solve MFA more reliably than trying to automate around it. This applies as much to OTP-based MFA on Salesforce as it does to any other MFA implementation.
Lightning components need component-aware automation logic, not generic selectors borrowed from standard HTML testing.
Selector healing should let you patch, not rebuild. The ability to update a single broken selector — rather than re-record an entire test — is what keeps a long-lived suite maintainable.
Automate the workflow, not just the page. Multi-screen, end-to-end coverage catches issues that isolated page-level tests would miss entirely.

We do this kind of testing work for teams dealing with Salesforce's dynamic DOM, Lightning components, or MFA that keeps breaking their automation. If you're wrestling with something similar, CloudQA is worth a look.

Why Automating Shopify Admin is a Nightmare (And How We Solved It)

CloudQA — Mon, 05 Oct 2026 03:57:50 +0000

Shopify is one of the harder platforms to automate reliably, and not for the reasons most teams expect.

Creating a product or syncing a feed isn't complex logic. The hard part is everything around it. This post breaks down how the QA team at CloudQA built stable, unattended end-to-end automation for AdNabu, a Shopify app for product feed and catalog automation, and the design decisions that made it work.

The four problems

Before a single meaningful test could run consistently, four separate problems had to be solved.

1. Session timeouts. Scheduled or long-running suites need a session that's still valid when they execute. Logging in fresh on every run is slower, and every login is another point of failure.

2. MFA prompts. MFA exists specifically to interrupt automated, unattended access. Hitting a challenge mid-run doesn't just fail one test, it stalls the whole pipeline behind it.

3. Shadow DOM. Shopify's admin encapsulates component internals in shadow roots. Standard DOM queries can't see inside them, so conventional selector strategies can't reach a meaningful portion of the UI.

4. Frequent UI changes. Shopify ships updates on its own schedule. Single-attribute selectors break whenever markup shifts, even when user-facing behavior hasn't changed.

Any one of these is manageable. Together, they make stable, repeatable, unattended automation genuinely difficult.

The approach

1. Isolate the environment first

Instead of running Shopify–AdNabu tests on shared infrastructure, the team set up a dedicated virtual machine for this workflow.

That removed a whole class of flaky-test causes: resource contention, state bleed from unrelated runs, and configuration drift from other projects. The VM's config, browser state, and session data belong exclusively to this suite, so when something fails, it's a Shopify–AdNabu problem and not an artifact of something else running alongside it.

2. Solve login and MFA once, not every run

Scripting around MFA on every run is fragile, and for many MFA implementations it isn't reliably automatable at all. So instead of fighting it, the team used a dedicated Chrome Debugger Profile that retains an already-authenticated Shopify session.

The profile persists login state between runs, so the automation attaches to a session that's already past the MFA challenge. One mechanism fixes both the session-timeout and MFA problems: as long as the profile's session is valid, tests run against a live, authenticated Shopify instance with no login step at all.

Here's a generic sketch of the pattern (illustrative, not CloudQA's code):

# Launch Chrome once with a dedicated profile and remote debugging enabled.
# Log in manually (including MFA) in this window one time.
google-chrome \
  --remote-debugging-port=9222 \
  --user-data-dir="$HOME/qa-profiles/shopify-adnabu"

// Playwright: attach to the already-authenticated browser instead of logging in
import { chromium } from 'playwright';

const browser = await chromium.connectOverCDP('http://localhost:9222');
const context = browser.contexts()[0];
const page = context.pages()[0] ?? await context.newPage();

await page.goto('https://admin.shopify.com/');
// No login step. The session is already valid.

The trade-off: you now own the lifecycle of that session. Plan for what happens when it eventually expires, such as an alert for a manual re-auth, rather than discovering it through a wall of red tests.

3. Handle Shadow DOM deliberately

Standard selectors stop at the shadow host. The team built custom logic for piercing shadow roots: first identifying which parts of the Shopify admin use Shadow DOM, then writing selector and interaction code that traverses into those shadow trees instead of stopping at the host element.

The result is automation that can click, type into, and read values from elements that would otherwise be invisible to the test framework.

If you're using Playwright, its CSS and text locators pierce open shadow roots by default. Here's a generic example of doing it by hand, useful when your tooling doesn't (illustrative):

// Walk into open shadow roots to find an element
function deepQuery(root, selector) {
  const direct = root.querySelector(selector);
  if (direct) return direct;

  for (const el of root.querySelectorAll('*')) {
    if (el.shadowRoot) {
      const found = deepQuery(el.shadowRoot, selector);
      if (found) return found;
    }
  }
  return null;
}

const saveButton = await page.evaluateHandle(
  () => deepQuery(document, 'button[type="submit"]')
);

Note that closed shadow roots can't be traversed this way, which is another reason to map out where Shadow DOM is used before you build your suite.

4. Self-healing selectors

To avoid rewriting tests every time Shopify ships a UI change, the team implemented fallback and self-healing selector logic that identifies elements using multiple properties instead of a single brittle reference. When the primary selector stops matching (an attribute changed, an ID shifted, markup was restructured), the automation falls back to alternate identifying properties to find the same functional element.

A minimal version of the idea (illustrative):

async function resilientLocate(page, strategies) {
  for (const { name, selector } of strategies) {
    const locator = page.locator(selector);
    if (await locator.count() === 1) {
      if (name !== 'primary') {
        console.warn(`Primary selector failed; healed via "${name}"`);
      }
      return locator;
    }
  }
  throw new Error('Element not found with any strategy');
}

const addVariant = await resilientLocate(page, [
  { name: 'primary',   selector: '[data-test-id="add-variant"]' },
  { name: 'role+text', selector: 'role=button[name="Add variant"]' },
  { name: 'aria',      selector: '[aria-label="Add variant"]' },
]);

Self-healing shouldn't hide real problems. The point is to stop cosmetic changes from being mistaken for failures, so the team's attention goes to real behavioral changes. Logging every heal (as above) keeps the healing visible instead of silent.

What got automated

With a stable environment, persistent auth, and resilient selectors in place, the team covered the workflows that matter most to AdNabu:

Product creation: building listings end to end
Variants: size, color, and other option combinations
Images: uploading and associating product imagery
Pricing: setting and updating price data
Google Feed sync: verifying product data syncs to the Google Shopping feed
AI-generated descriptions: validating AI-produced description content

Image uploads, feed syncs, and AI generation all complete on their own timelines, so the suite uses dynamic waits instead of fixed sleeps. Tests proceed as soon as an operation genuinely finishes, not on a guessed timer.

// Instead of: await page.waitForTimeout(10000);
await expect(page.locator('[data-status="synced"]'))
  .toBeVisible({ timeout: 60_000 });

The outcome

The combination of an isolated environment, persistent authentication, Shadow DOM handling, and self-healing selectors produced a suite that runs unattended and stays stable through Shopify's routine UI updates. It doesn't need manual re-login, doesn't stall on MFA, and doesn't need constant selector rewrites.

Key takeaways

Isolate before you optimize. A dedicated environment eliminates a whole class of flakiness before you write any selector logic.
Solve authentication once, not every run. Persisting a session removes login and MFA as a per-execution failure point.
Shadow DOM needs a strategy, not a workaround. Plan for it up front instead of discovering it mid-project.
Self-healing buys time, not immunity. It absorbs UI churn so you can focus on genuine regressions.

These patterns aren't Shopify-specific. Any modern SaaS admin with MFA, web components, and a fast release cadence presents the same problems, and the same playbook applies.

This post is based on a CloudQA case study of their work with AdNabu.