DEV Community: qa

The Defect Life Cycle: From New to Closed

keyur shah — Fri, 06 Nov 2026 11:42:33 +0000

Every software team eventually asks the same question: “Where is this bug right now?”

Regardless of whether you track work in Jira, Azure DevOps, or a simple spreadsheet, a defect moves through a predictable set of states. Understanding these states, who owns them, and the best practices around them keeps the flow smooth and the product stable.

The Core States

State	What Happens	Owner
New → Assigned	A tester or user discovers an issue, logs it with a clear description, and the triage lead hands it to a developer.	QA creates the record; the triage lead assigns it.
Fixed → Retest	The developer believes the code change resolves the problem and marks the defect as Fixed. The QA team then picks it up for verification.	Developer marks Fixed; QA takes it to Retest.

These two transitions are the heartbeat of the cycle. “New” is the entry point, and “Retest” is the first real test of whether the fix works.

Retest Outcomes

State	What Happens	Owner
Verified → Closed	QA confirms the defect no longer reproduces using the original steps. The defect is marked Verified and then Closed.	QA verifies; the product owner may give final sign‑off before closure.
Reopened	The fix fails under the original repro steps, or a regression is discovered. The defect goes back to Assigned for another round.	QA reopens; the defect returns to the developer.

A high reopen rate often signals flaky code, insufficient test coverage, or unclear requirements. Tracking this metric helps pinpoint weak components.

The Other Exits

State	What It Means
Duplicate	The same issue has already been logged under a different ID. The duplicate is linked and closed to avoid waste.
Won’t Fix / Deferred	The team decides the risk is acceptable, or the fix is postponed to a later release. This decision should be documented and reviewed regularly.

These exits prevent the backlog from becoming a dumping ground. They also ensure that every decision is visible to stakeholders.

Who Owns What

Role	Responsibilities
QA	Owns New, Retest, and Verified. QA finds defects, adds reproducible steps and evidence, retests fixes, and confirms closure.
Dev	Owns Assigned and Fixed. Developers diagnose the root cause, implement a fix, and mark the defect as Fixed.

Clear ownership eliminates hand‑off confusion and speeds up the cycle.

Do / Avoid

Attach reproducible steps and screenshots at New; don’t wait until later.
Re‑verify using the exact original steps during Retest; a quick glance can miss subtle regressions.
Track the reopen rate per component; treat spikes as a quality signal that needs investigation.

Avoid

Closing a defect without an independent retest; it creates hidden technical debt.
Letting “Won’t Fix” become a permanent sink; schedule periodic reviews to reassess.
Skipping severity or priority logging at creation; it hampers triage and release planning.

Understanding and respecting the defect life cycle keeps teams aligned, reduces waste, and delivers more reliable software.

Master the flow, and the bugs will flow through it predictably.

Book a call

How to Review AI-Generated Test Cases Before You Trust Them

Craig Brown — Fri, 06 Nov 2026 11:41:23 +0000

I'm the only QA engineer on a large development team, so I use AI a lot. ChatGPT, Claude and Copilot all help me get a first draft of test cases down far faster than I could type them.

But here's the thing I've learnt the hard way: AI drafts look more complete than they are. They're well formatted, confidently worded and neatly numbered. That polish makes it very tempting to skim, nod and paste them into the test management tool.

[MY REAL EXAMPLE]

The AI's job is to draft. My job is to verify, not rubber-stamp. Below are the seven checks I run on every AI-generated set of test cases before I trust any of them.

Trace every test back to a requirement

Every test case should exist because of something in the story, spec or acceptance criteria. If I can't point to where a test came from, I don't know whether it's testing the product or testing the AI's imagination.

What to look for: a test like "Verify the user can export results to PDF" when the story only mentions CSV export. It reads perfectly sensibly. It's just not in scope.

Quick tip: ask the AI to add a "Source" line to each test quoting the exact requirement it covers. Then check those quotes against the real document — the AI can misquote too.

Hunt for invented details

This is the one that catches people out most. When the AI doesn't know a detail, it often fills the gap with something plausible: a field name, an error message, a validation rule, a timeout, a role name.

What to look for:

Field labels that don't match the UI ("Email Address" when the screen says "Work email")
Exact error message text nobody gave it
Business rules stated as fact ("accounts lock after 5 failed attempts") that appear nowhere in the requirements

Quick tip: highlight every specific value in the draft — numbers, strings, names — and confirm each one against a source. Anything you can't confirm becomes a question for the product owner, not an expected result.

Check negative and edge cases

AI drafts lean heavily towards the happy path. You'll often get six variations of "valid input succeeds" and one token "invalid input shows an error".

What to look for: missing coverage for empty fields, boundary values, maximum lengths, special characters, duplicate submissions, expired sessions, permissions for different user roles, and what happens when a dependency is slow or unavailable.

Quick tip: after the first draft, ask specifically: "List the negative, boundary and error-handling cases that are missing from these tests." Then apply your own judgement — you know where this system has broken before. The AI doesn't.

Make sure expected results are testable

An expected result should let two different testers reach the same pass/fail verdict. AI loves vague outcomes that sound like results but can't actually fail.

What to look for: phrases like "the system works as expected", "the page loads correctly", "an appropriate message is displayed" or "data is saved successfully".

Quick tip: for each expected result, ask yourself "how would I prove this failed?" If there's no clear answer, rewrite it as something observable: a specific message, a specific state, a specific record in the database.

Remove duplicates and padding

Ask for twenty test cases and you'll usually get twenty, whether the feature needs them or not. Volume isn't coverage, and padded suites cost real time to run and maintain.

What to look for: tests that differ only in wording, tests that check the same rule with an equivalent input, and filler like "Verify the page title is displayed" that adds nothing to the risk you're covering.

Quick tip: group the tests by the rule or behaviour they cover. Where two tests land in the same group with no meaningful difference in input or risk, merge them or delete one.

Check for sensitive data in prompts and test data

This applies in both directions. What did you paste into the prompt, and what has the AI put into your test data?

What to look for:

Real customer names, emails or account details copied from production into a prompt
API keys, connection strings or internal URLs in pasted logs or config
Generated test data that looks like real personal data and might end up in shared environments or screenshots

Quick tip: anonymise before you prompt, and use clearly fake data in tests (for example, addresses on a reserved domain like example.test). Check your organisation's policy on which AI tools are approved for which kinds of data.

Record what you verified

If a test case was drafted by AI and reviewed by you, that should be visible. It protects you, helps your team understand how much to trust the suite, and matters a great deal in regulated environments where you may need to show your working.

What to look for: test cases with no indication of where they came from, who checked them, or what was changed.

Quick tip: keep it light. A short review note — "AI draft, reviewed against AUTH-112, two invented rules removed, three negative cases added" — is enough. You don't need an essay; you need a trail.

Before and after: one test case

Here's a typical AI draft for a password reset story:

text
TC-014: Verify password reset works
Steps:

Go to the login page
Click "Forgot Password"
Enter a valid email
Click Submit Expected: User receives a reset email and the system works as expected.

And the same test after review:

text
TC-014: Reset email is sent for a registered address
Source: AUTH-112, acceptance criterion 1
Preconditions:

Account exists for test.user01@example.test
Test mailbox is accessible in the QA environment Steps:
On the login page, select "Forgot password?"
Enter test.user01@example.test
Select "Send reset link" Expected:
Confirmation message matches the copy in AUTH-112
Exactly one reset email arrives in the test mailbox
The email contains a link to the reset page for that account Review notes:
AI draft, reviewed by CB
Link label corrected to match UI
Invented message "Check your inbox!" removed
Unregistered email and expired link split into TC-015 and TC-016

The reviewed version is traceable, uses real labels, has expected results that can fail, splits out the negative cases, and leaves a record of what changed.

Wrapping up

AI has made me faster at drafting tests, and I'm not going back. But speed only helps if what comes out is right. The polish on an AI draft is not evidence of quality — it's just formatting.

Treat every AI-generated test case as a first draft from a confident colleague who hasn't read the spec. Trace it, question it, tighten it, and write down what you checked. That's where the actual testing happens.

AI drafts. You verify.

I've put together a kit with 38 QA prompts and a 15-point checklist for this: https://trustworthy-ai-testing.lovable.app/

The first four weeks of a QA course in Nepal, and why automation waits

Saarathi Academy — Fri, 06 Nov 2026 11:09:37 +0000

The first month of a good QA course has no test automation in it. You spend four weeks learning what to test, how to report a bug so it gets fixed, and how to check an API and a database.

Automation comes after that, because a script only repeats the checks you chose. If you choose weak checks, the script runs weak checks faster.

What does a QA engineer actually do?

QA (quality assurance) means checking that software works before users get it. A QA engineer reads what a feature should do, decides how to test it, finds where it breaks, and writes that down clearly enough for a developer to fix it.

Most of that is thinking and writing, not coding. Code helps later, when the same checks must run on every change. So the first month builds the thinking, in this order.

Week 1: Where does testing fit in building software?

Start with how software gets made. Learn the steps a feature goes through, from the idea to the release, and where testing sits in each step. Learn the main kinds of testing too: checking one small part, checking parts together, and checking the whole app as a user would.

Add Git basics this week. Later, your test code lives in Git, and employers will look at your GitHub.

Then practise on something built for testers. Practice Software Testing is a free demo shop. Its makers keep one version with bugs added on purpose, so you know there is something to find. Pick one page, like login, and write down everything it must do.

Week 2: How do you choose which inputs to test?

You cannot test every possible input, so you learn to pick the few that matter. This is called test design, and it is what separates testing from clicking around.

Two techniques do most of the work. Equivalence partitioning splits inputs into groups that should behave the same, and tests one value from each group. Boundary value analysis then tests the edges of each group, because that is where code often breaks.

Here is a simple case. A month field accepts 1 to 12. You test 0, 1, 12 and 13, which are the values on both sides of each edge. For a Nepali mobile number field, you would test 9, 10 and 11 digits, not fifty random numbers.

This is also a good week to start JavaScript on the side, a little each day. You will need it in week five, and it is easier to learn slowly than in a rush.

Week 3: What makes a bug report good enough to fix?

A bug that nobody can reproduce does not get fixed. Mozilla's bug writing guide calls the steps to reproduce "the most important part of any bug report".

A good report has four parts:

a short summary that names the problem,
numbered steps to reproduce it,
what you expected and what actually happened,
the browser, device and version you used.

Mozilla gives two more rules worth keeping. Separate what you saw from what you guess. Write one bug per report.

If English is your second language, this is good news. Short numbered steps work better than long paragraphs. A developer reading "3. Click Pay. 4. Page shows Rs 0" understands it at once.

Learn two words this week too. Severity is how bad the bug is. Priority is how soon it should be fixed. A typo on the home page has low severity but can still have high priority.

Week 4: Why do testers need Postman and SQL?

Many bugs hide behind the screen. The page says "Saved", but nothing reached the database. The form shows an error, but the server crashed underneath it.

Postman lets you send a request straight to an API (the part of an app that sends and receives data) and read the answer. Every answer has a status code. Codes from 200 to 299 mean success, 400 to 499 mean the request was wrong, and 500 to 599 mean the server failed, as MDN explains.

That gives you an easy first test. Send an empty or wrong value. A good API answers with a 400 and a clear message. A 500 means the server did not know how to handle it, and that is a bug to report.

SQL does the same job for data. You place an order in the app, then run one SELECT query to check that the row exists with the right amount. Testers who can do this find bugs that clicking alone never shows.

Is an old laptop enough for the first month?

Yes. These four weeks need a browser, the Postman desktop app, a text editor and Git. None of them are heavy, and the practice shop runs online, so there is nothing big to install.

Heavier tools come later, with automation. By then you will know if QA suits you.

What can you try tonight?

Open the buggy version of the practice shop and give yourself 30 minutes. Find one bug. Write it up with the four parts above, as if a developer you have never met will read it.

Then install Postman and follow its guide to send your first request. Look at the status code, then change the address so it breaks, and look again.

Saarathi Academy in Old Baneshwor, Kathmandu teaches QA in this order in its Software Quality Assurance course: manual testing in weeks 1 to 3, Postman and SQL in week 4, then Playwright automation. The week-by-week syllabus is public before you pay.

nexus studio mobile testing

nexus studio — Thu, 05 Nov 2026 15:38:10 +0000

One test, three phones, and each one files its own evidence.

Nexus StudioQA runs the same test on emulated profiles (Chrome on a Pixel 5, WebKit on an iPhone 13, a low-end phone with the CPU throttled) and History lines them up per identity. For a real Android phone, a guided setup walks through six steps over USB, and the phone's screen is mirrored on your PC while the test runs. No device farm and no minutes to buy.

Limits, plainly: real phones are Android only. iPhone and iPad are covered through WebKit emulation, not physical devices.

Cypress Email Tests Need a Causal Trace

Silviu Technology — Thu, 05 Nov 2026 14:23:39 +0000

A Cypress email test can tell you that a verification link was not found. It usually cannot tell you why. The message may have arrived late, the test may have searched the wrong inbox, or the application may never have requested the email at all. Those failures look very similar when the only artifact is a timeout.

For reliable Testing, I treat each email assertion as a small causal investigation. The test should connect the user action, the backend event, the inbox observations, and the final link click. A pass or fail is the conclusion; the causal trace is the evidence that makes the conclusion useful.

This approach works with a temporary inbox, a local mail server, or a disposable email generator. The provider matters less than the evidence contract around it. Search terms such as temp mail so may describe the kind of test data a team is looking for, but they do not explain a failing run.

Why a pass or fail result is not a cause

Imagine a signup test that clicks Create account, waits 30 seconds, and fails to find “Verify your email.” On the next retry it passes. The team may call this flaky, but several different causes are still possible:

The first request was never queued.
The email arrived before the polling window started and was filtered out.
A previous run left a message with the same subject.
The inbox API returned a temporary error.
The link was present, but the browser used a different account.

Retries can hide the distinction. The second attempt changes timing, inbox state, and often the test data too. If the log only says “expected email to exist,” there is no useful comparison between attempts. Thats why I want a trace that survives both a pass and a failure.

The event timeline to capture

An email assertion does not need a huge observability platform. Four timestamps and a few identifiers are enough to start:

actionAt: when Cypress submitted the form.
requestId: the application request or correlation ID, if one is exposed.
pollStartedAt: when the inbox query began.
messageId: the message accepted by the test.

I also record the inbox address, the test run ID, the message subjects observed, and the reason each candidate was rejected. This makes the timeline readable:

14:02:11.204 actionAt       signup submitted
14:02:11.260 requestId      req_7f2c
14:02:11.310 pollStartedAt  inbox polling started
14:02:12.018 observed       msg_91, subject="Welcome"
14:02:13.477 accepted       msg_92, subject="Verify your email"
14:02:14.006 clicked        verification link

The trace should not store full message bodies or tokens in ordinary CI logs. Save a redacted artifact instead. Email testing is easier to debug when it is also careful with test data.

If the product has recovery links, it is worth checking the related security behavior too. A revocation plan for recovery links is useful context when an end-to-end test follows a link beyond the inbox.

A Cypress helper for causal evidence

I keep polling and evidence collection in one task. Individual specs then state what they are testing instead of rebuilding timing logic each time.

type EmailTrace = {
  inbox: string;
  actionAt: string;
  pollStartedAt?: string;
  observed: Array<{ id: string; subject: string; receivedAt: string }>;
  rejected: string[];
  acceptedId?: string;
};

function waitForVerification(inbox: string, actionAt: Date) {
  const trace: EmailTrace = {
    inbox,
    actionAt: actionAt.toISOString(),
    observed: [],
    rejected: [],
  };

  return cy.task('pollInbox', { inbox, since: actionAt.toISOString() })
    .then((messages: MailMessage[]) => {
      trace.pollStartedAt = new Date().toISOString();

      for (const message of messages) {
        trace.observed.push({
          id: message.id,
          subject: message.subject,
          receivedAt: message.receivedAt,
        });

        if (!message.subject.includes('Verify your email')) {
          trace.rejected.push(`${message.id}: subject mismatch`);
          continue;
        }

        trace.acceptedId = message.id;
        return { message, trace };
      }

      throw new Error(`No verification message; trace=${JSON.stringify(trace)}`);
    });
}

In the spec, capture actionAt immediately before submitting the form, then attach the returned trace to the Cypress test artifact. Do not use “latest message” as the only rule. Filter by time, subject, recipient ownership, and a request or run identifier when the application can include one.

The small detail that saves time is recording rejected candidates. Without it, a developer sees that no message matched. With it, they see that two welcome emails arrived and both failed the ownership check. That is a different bug, and it points to fixture isolation.

How to classify the failure

After a failed run, classify the trace before changing the timeout. I use four buckets:

Application: no request or queue event exists after the browser action.
Delivery: the request exists, but no message appears within the agreed window.
Fixture: a message appears, but it belongs to another run or recipient.
Assertion: the right message is present, but parsing or link validation is wrong.

This classification also helps compare Cypress with other browser suites. For example, teams often need explicit email states in signup flows before a UI can distinguish “queued” from “delivered.” If the UI has only a spinner, the browser test may be forced to guess too.

A common note in a rushed test is temp org mail or temp gamil com. Those phrases might be useful search variants, but they are not a reason to widen a polling loop. First identify which bucket the trace supports. Then make the smallest fix.

A repeatable checklist

Before merging an email test, check that it:

creates or leases an inbox for the run;
records the browser action time;
filters messages newer than that action;
logs rejected candidates without exposing message contents;
verifies recipient or run ownership;
stores a redacted trace on failure and on retry;
distinguishes delivery delays from assertion mistakes.

The list is intentionally plain. A reliable test is not the one with the most waits. It is the one that leaves enough evidence for the next person to explain what happened, even when the run is inconsistant or a provider responds slow.

Q&A

Should every email test use a unique inbox?

For parallel or retry-heavy suites, yes when the provider makes that practical. If unique inboxes are expensive, use a stable inbox with a strict run identifier and a message ownership check. Shared inboxes without either rule create confusing false positives.

How long should Cypress poll?

Use the delivery expectation of the environment, then make the timeout visible in the trace. A longer timeout can reduce noise, but it cannot repair a missing request or a wrong recipient. Increasing it blindly is usually a delay, not a diagnosis.

Do I need a full event-tracing system?

No. Start with a structured JSON artifact containing the action timestamp, request ID, inbox, observed message metadata, rejection reasons, and accepted message ID. That is enough to turn many vague email failures into a concrete next step.

Decentralize ownership, not load testing

Gatling.io — Thu, 05 Nov 2026 10:34:09 +0000

The release candidate is ready, the functional tests are green, and the product manager has a date. Then someone asks whether it has been load tested. The performance team's next slot is in two weeks.

This is the traditional model: a small group of specialists, a shared pre-production environment, and a big load test at the end, used as a gate. When it works, it catches problems before customers do. When it doesn't, it catches them too late to fix, or it gets skipped. Contractors told Congress that end-to-end testing of Healthcare.gov began two weeks before launch. MongoDB's performance team has described, in a peer-reviewed paper, how its own end-of-cycle testing led to delayed releases or regressions shipped to customers.

The usual answer is to "shift left" and hand testing to developers. The evidence says that is only half right. Our conclusion, and how our largest customers operate: decentralize ownership of performance, and centralize the platform that supports it.

Key takeaways

Performance testing strategy

Takeaway	Why it matters
Central, late-cycle load testing is risky	Tests become rare events, feedback reaches the wrong person late, and late gates get overruled.
Handing tests to developers isn't enough	Unsupported teams write few, flawed tests and miss system-level bottlenecks.
Teams own performance, the platform supports them	Teams own objectives, scenarios, and fixes; the platform owns infrastructure, standards, and system-level tests.
Six practices make it work	Tests in the repo, agreed pass/fail, shared methodology, run comparison, central governance.
It works at scale	Sophos runs without a central QA team; Intuit grew from fewer than 100 to 3,000+ Gatling users.
It isn't for everyone	Small teams, tightly coupled systems, and missing CI/CD change the answer.

What a central team gets right, and what it costs

A dedicated performance team exists for good reasons: specialists who can model a workload and tell a real regression from noise, one consistent methodology, a view of the whole system, and tight control over who can point heavy traffic at what. Those strengths are real. Here is what they cost as the organization grows.

Testing becomes an event: When a load test means booking the shared environment and gathering the specialists, it gets scheduled rather than done. LoginRadius described the result: "That pushed performance testing to the back burner; it only ran when absolutely necessary." Yet DORA's definition of continuous integration includes developers getting feedback from performance tests every day.
Feedback arrives late, and to the wrong person: A regression introduced in week one surfaces in week ten, mixed with dozens of other changes. The person who finds it rarely caused it, so it lands as a ticket about code its author has moved on from. MongoDB described exactly this diagnosis burden before moving its tests into CI.
Teams wait in line: DORA treats testing on demand, without waiting for an integrated environment, as a core capability of high-performing teams. Intuit, describing its own specialist model, said every team needing a load test joined a queue. With finite capacity, a central team covers what is most visible. At IMA Group, much of the rest shipped untested.
Tests model last quarter's traffic: Big suites are rebuilt rarely, while users and software keep changing. Facebook's engineers made this point when they built Kraken to load test with live traffic: a capacity model is hard to keep current.
A late gate is easy to overrule: On September 30, 2013, a stress test of a key Healthcare.gov component reached 1,100 users before response times became too high. The target was 10,000. The site launched the next day. Officials later disputed how representative that test was, which is the point: one late result is easy to argue with.

These costs compound. Hard-to-run tests happen rarely, rare tests cover big batches of change, big batches are slow to diagnose, and late results get overruled. Adding specialists to the central team doesn't break that loop.

Where decentralization goes wrong

Handing every team a load testing tool doesn't solve this. The research is clear about the failure modes.

Developers left alone write few tests, and many are flawed: In a study of 111 open-source Java projects, the median project had 246 lines of performance test code against 3,980 of functional tests. A study of 123 projects found benchmarking mistakes in widespread use.
The tooling is hard: A SPEC Research Group survey found that tool complexity holds back performance testing in DevOps.
Component tests miss system behavior: Microservice baselines are noisy, and some bottlenecks only appear when dozens of services are loaded together. That is why Shopify, whose teams own their services, still runs coordinated platform-wide scale tests before Black Friday.
Autonomy isn't accountability: At Spotify, by one former engineer's account, the alignment and accountability parts of the squad model were never completed.

Centralized execution isn't the problem either: Meta runs a shared performance platform, ServiceLab, across its services. Moving the test is not the same as moving the ownership.

The model: central platform, distributed ownership

Who owns what in decentralized performance testing?

Shared ownership

Product teams own	The platform owns
Performance objectives and pass/fail criteria	Load generators, environments, secrets and permissions
Test scenarios, versioned with their code	Templates, standards and methodology
Fast regression checks in CI	Governance, budgets and cross-team visibility
Fixing what the tests find	System-level, capacity and peak-event tests

Google's SRE model keeps services with product teams while SRE owns capacity planning. Netflix built its canary analysis tool, Kayenta, centrally, and service owners choose the metrics it judges. Team Topologies calls this a platform team plus an enabling team. Specialists don't disappear. They stop being a queue and start building the paved road.

Six principles, and how Gatling Enterprise Edition supports them

Each failure mode above has a countermeasure. Here is how we see them, and the Gatling Enterprise Edition capability behind each.

1. Tests live in the service repo, in the team's language: Friction is a big reason developers write few performance tests. Gatling tests are code in Java, JavaScript, TypeScript, Kotlin or Scala, reviewed in pull requests like any other change. Intuit ships a pre-configured performance test folder with every new repo. EPI Company writes its tests in Kotlin because, says Ulrich Winter, "that really lowers the barrier for developers to contribute."

2. The platform takes the hard shared work: Gatling Enterprise Edition runs managed load generators, or Private Locations inside your own network, so product teams never maintain test infrastructure. At Nickel and InPost, product teams own their scenarios while a platform team handles environments, secrets and permissions. "We stopped maintaining boxes," says InPost's Mateusz Piasta.

3. Pass/fail is agreed before the test runs: Thresholds live in the simulation as assertions, such as forAll().responseTime().percentile(95).lt(300), or as SLOs set in Gatling Enterprise Edition, which take precedence when both exist. Sophos teams test against service-level agreements set by product management. Tikamoon's rule is shorter: "No Gatling run, no production."

4. Methodology is standardized centrally: If one team uses a closed injection model and another an open one, their p95 numbers can't be compared. The platform team should settle conventions like this once, in templates. Campaigns in Gatling Enterprise Edition score every test on methodology, objectives and stability over its last three runs.

5. Runs are compared over time: Noisy environments make a single run against a fixed threshold misleading. Run comparison, the trends view and AI Trend Analysis show where a regression appeared. That is how InPost caught a doubling of requests per second, caused by a needless HTTP-to-HTTPS redirect, before it reached production.

6. Governance stays central, and system-level tests stay coordinated: Gatling Enterprise Edition gives each team its own credit quota, roles and isolated workspace, while platform owners keep visibility across all of them. Attentive's service owners each name a contact for cross-service game days that run every morning through the API. InPost's five-day simulation of the full parcel journey found bottlenecks "from code to cables."

What it looks like in practice

Sophos: no central QA team

When Sophos moved to microservices, it chose not to build a central QA or performance team. Each microservice team tests its own APIs against business-defined service-level agreements and reuses simulations across teams, while leadership tracks results and regressions in shared dashboards. Around 200 people now use the platform, with 100+ simulations covering 75 to 80% of backend APIs. Seeing their own services under load, teams also found endpoints that were wasting cloud resources.

Intuit: the same model at 8,000 engineers

Intuit's specialist model meant late feedback, regressions in production, a queue for every team, and around 30 fragmented tools. Principal Engineer Chaitanya Bhatt rebuilt it as a platform: test scaffolding in every repo, CI/CD onboarding, pre-built templates, and central governance. The Gatling Enterprise Edition API lets Intuit provision teams, simulations and roles from its internal developer portal. Gatling users grew from fewer than 100 in 2018 to more than 3,000, running 80,000+ tests a year with 97% coverage of critical services. Intuit didn't remove the center. It turned it into a platform.

Other ways to split it

Attentive has no testing department, but a Developer Experience team runs the shared tooling. IMA Group and SNCF Connect & Tech both replaced central testing units with team ownership. In the other direction, Aircall grew a dedicated performance department as demand rose, and Criteo runs five performance engineers supporting 70+ colleagues. The goal isn't to remove specialists. It's to stop them being the only place testing happens. More examples are in our customer stories.

When not to decentralize

Small organizations: Without people to staff a platform, decentralizing just means nobody runs the tests.
Tightly coupled systems: Test ownership won't fix architectural coupling. The system-level test is the one that matters.
No CI/CD or observability: Teams get numbers they can't act on. Build those first.
Regulated environments: These need an auditable gate, but that fits a federated model. At Nickel, a BNP Paribas subsidiary, product teams own their scenarios, load runs from Private Locations inside Nickel's network, and results go through a mandatory Change Advisory Board review.

How to start

Make the platform self-service through an API and CI integration.
Move scenarios into team repos, starting with a few willing teams and templates.
Add short checks to CI, and keep long tests for later stages.
Standardize injection models, percentiles and pass/fail conventions.
Keep coordinated capacity and peak-event tests with a named owner.

Measure results, not adoption: time from commit to regression detection, performance incidents in production, and time to repair them. And pilot first, because DORA's 2024 report associates new internal platforms with an initial dip in throughput and stability.

Ownership is the point

A central team that owns every test becomes a queue. Developers who own tests without standards or support produce numbers nobody trusts. Give each team ownership of its own performance, and give the platform everything teams can't do alone. As IMA Group's Mehdi Chilla puts it: "Performance testing should be part of the natural workflow, not an exceptional activity."

See how Intuit runs this across 8,000 engineers in Chaitanya Bhatt's 45-minute session, or compare Gatling Community and Enterprise Edition.

[

](https://gatling.io/intuit)

An AI agent tested Toolshop from just the URL: 23 issues, 8 real

Shahid Helal — Thu, 05 Nov 2026 10:18:44 +0000

Toolshop's sign-up page says a password needs at least 8 characters. The form enforces 8. The API enforces 8. Type 7 and the error message says the minimum is 6.

That's the best of 8 real bugs our AI testing agent found on Toolshop, the demo hardware store a lot of testers learn Playwright and Cypress on. It got only the URL. No test plan, no credentials, no hints.

This is Weekly Teardown 2. Each week we point TestOptim, the agent we build, at a public app and publish what it finds, including what it got wrong. Teardown 1 was ParaBank, whose admin console had no login.

The run

In 30 minutes the agent explored 17 pages and mapped 16 user flows and 29 business rules. It wrote 289 tests, ran 71 of them before the time limit, and flagged 23 issues.

A list of 23 issues isn't worth much if you have to check all 23 yourself. So I did, on the live site.

What held up

1. The password rule changed, and its error message didn't. The sign-up page says a password must "Be at least 8 characters long." The form validator enforces 8:

// sprint5/UI/src/app/auth/register/register.component.ts
PasswordValidators.minLength(8)

So does the API:

// sprint5/API/app/Http/Requests/Customer/StoreCustomer.php
Password::min(8)

But the error text lives in a separate translation file, assets/i18n/en.json, under fields.password.errors.minlength, and it still says "Password must be minimal 6 characters long."

The rule says 8 in three places and 6 in one. Each piece looks fine on its own, which is why this kind of drift is easy to miss in a manual review and easy to catch for an agent that reads the page, submits the form and compares the two.

2. The wrong error for a missing symbol. Leave the special character out and the error says "Password can not include invalid characters." Nothing is invalid; something required is missing. A user who reads that deletes characters, which is the opposite of the fix.

3. A menu link to a category that doesn't exist. Categories → Special Tools opens "There are no products found." The public API has no Special Tools category at all.

4. An empty cart breaks checkout. Open /checkout with nothing in the cart and nothing renders under the step bar. The console says why:

TypeError: Cannot read properties of null (reading 'cart_items')

5. A product that doesn't exist gives a blank page. Mistype a product URL and you get an empty page with a "Related products" heading instead of a "not found" message.

6. "Forgot password" says nothing. Enter an email, click "Set New Password", and the page stays as it was: no confirmation, no error.

7. "Show differences only" only half works. It filters the specifications, but rows that are identical for every product, like Brand, Category and Availability, stay on screen.

8. "2years". The product page prints the warranty as "2years". The comparison page prints the same value as "2 years".

What I threw out, and why

9 findings didn't survive the re-check. They're the more useful half of this post if you're evaluating any AI testing tool.

Fake payments "accepted". The agent rated a 1111-1111-1111-1111 card and a made-up gift card as critical. In a real store they would be. Toolshop has no payment processor, so on a demo it's expected. Same for an address lookup that invents a street for a made-up postcode.
The eco discount "never applied". This one was our mistake. The agent guessed the rule. Toolshop's code applies the 5% eco discount only when more than half the cart is rated A or B for CO₂, and the agent's test cart didn't qualify.
Design choices, not bugs: a contact form that disappears after sending, an order confirmation that shows only the invoice number, category pages without the home page's price filter, and a rentals card the agent expected to show a price.
No longer reproduces: the agent said an empty sign-up form showed 10 of its 12 errors. Today all 12 show.

6 more need an account to check (a stale cart total, an HTTP 500 on registration, the payment step), so I've left them out rather than guess.

What I took from it

The agent was strongest where people are weakest: a rule written down in several places that drifts in one of them, and pages that break in states nobody visits on purpose, like an empty cart or a mistyped URL.

It was weakest where it had to guess a business rule the site never states, or tell a demo's shortcuts from real bugs. Closing that gap is what we're working on next.

If you want to try it on your own app, give TestOptim your staging URL. It explores the pages, maps the flows and business rules, writes and runs the tests, and sends each bug with screenshots, repro steps and a severity to Slack, Jira or GitHub.

Got a public app for next week's teardown? Drop the URL in the comments.

Quality Assurance - Web Testing

Joy Odinaka — Thu, 05 Nov 2026 09:51:10 +0000

I have learned how to test features, write test cases, execute tests, and report bugs.

As a software developer transitioning into software testing, my next step is WebTesting.

What actually happens when I interact with a web application?

Say I click Login, search for a restaurant, or add food to my cart, what's happening behind the scenes?

1. How the Web Works

A web application involves communication between a client and a server.

A simplified flow looks like this:

User
  ↓
Browser (Client)
  ↓
HTTP Request
  ↓
Web Server
  ↓
Database
  ↓
HTTP Response
  ↓
Browser displays the result

For example, when I log in:

I enter my email and password.
The browser sends a request to the server.
The server validates the credentials.
The server responds with a result.
The browser displays the appropriate response.

As a tester, I need to understand what happens at each stage.

2. HTTP vs HTTPS

HTTP and HTTPS are protocols used for communication between browsers and servers.

The key difference?

HTTPS encrypts communication using TLS, helping protect sensitive information while it's being transmitted.

For example, when testing a login page, I need to verify that the application uses HTTPS rather than sending login credentials over an insecure HTTP connection.

3. Chrome DevTools

One of the most useful tools I am learning is Chrome DevTools.

It helps testers investigate what's happening behind the user interface.

Two important tabs:

Network Tab

Helps me inspect:

HTTP requests and responses
API endpoints
Status codes
Request and response headers
Response times
Failed network requests

Console Tab

Helps me identify:

JavaScript errors
Warnings
Runtime issues
Other messages that may help investigate defects

For example, if restaurant search returns an empty result, I can inspect the Network tab to see whether the request succeeded or failed.

4. Web UI Testing

This is where I check what users actually interact with.

Examples:

Are buttons working?
Are input fields accepting the correct values?
Are error messages displayed correctly?
Are links navigating to the right pages?
Does the layout work on mobile and desktop?
Are images loading correctly?
Are forms validating user input?

A page looking good doesn't guarantee it works correctly.

Appearance and functionality both matter.

5. Cookies, Sessions, Local Storage and Cache

These are important concepts because web applications often need to remember information.

Concept	What it does
Cookies	Store small pieces of data in the browser
Sessions	Help maintain a user's interaction state
Local Storage	Stores data in the browser across sessions
Cache	Stores resources to improve loading performance

For example, when I log into QuickBite, I expect my authenticated session to be maintained as I navigate between pages.

Things I should test:

Does the user remain logged in after refreshing?
Does logout properly end the session?
What happens when a session expires?
Is sensitive information stored insecurely?
Does clearing browser data affect the application as expected?

6. What Should I Look Out for When Testing Web Applications?

A web application should be tested beyond the working features.

Some important areas include:

Functionality: Do features work as expected?
Usability: Can users navigate the application easily?
Compatibility: Does it work across different browsers and devices?
Reliability: Does it behave consistently?
Performance: Does it respond within acceptable times?
Security: Are user data and application resources properly protected?

What I am Learning

Web testing has made me realize that there's much more to an application than what appears on the screen.

A button click can trigger a request, communicate with a server, interact with a database, and return a response.

Understanding these interactions as a QA tester helps me investigate problems more effectively.

You don't just plan to only test what you see. You also want to understand what's happening behind it.

What's Next?

I am moving into API Testing.

I'll be learning how to test the communication between applications and servers directly, without relying entirely on the user interface.

I'm learning QA and I'm taking you with me.

AI Can Write Tests. But Are We Testing the Right Things?

Rimon Nassory — Thu, 05 Nov 2026 08:19:00 +0000

AI is changing how we approach software testing. But generating more tests doesn't necessarily mean delivering better software.

AI Is Making Test Automation Faster

Over the past year, I've been exploring how AI can improve our QA processes, especially when working with Playwright and automated regression testing.

And one thing is clear: AI can be incredibly helpful.

It can generate test scenarios, write automation scripts, suggest edge cases, and help investigate failing tests.

Tasks that previously took hours can sometimes be completed much faster.

But I've started asking myself an important question:

Are we actually improving software quality, or are we just generating more tests?

More Tests Don't Always Mean Better Quality

Imagine asking AI to generate tests for a new feature.

It produces 20 automated tests. They all pass. CI is green.

Looks great, right?

But what if the requirements were incomplete?

What if an important business rule was never discussed?

Or what if the AI generated tests based on how the implementation works, rather than how it should work?

You can end up with a perfectly green test suite that validates the wrong behavior.

And that's where I believe QA becomes even more important.

QA Needs to Be Involved Earlier

Traditionally, QA is often involved when a feature is ready for testing.

But with AI accelerating development, waiting until the end becomes even more risky.

If developers can generate working implementations faster, unclear requirements can become working code just as quickly.

That's why I believe QA should be involved before development begins.

We need to ask questions like:

What is the expected behavior?
What could go wrong?
Which edge cases are we missing?
What are the risks?
How do we know the feature is actually correct?

Once we understand these things, AI becomes much more powerful.

Instead of asking AI to guess what needs testing, we can give it clear expectations and let it help us turn them into reliable automated tests.

AI Should Support QA, Not Replace Judgment

I'm also interested in concepts such as LLM-as-a-Judge and self-healing tests.

Both have exciting potential, but they introduce new questions.

Should an AI automatically fix a failing test?

How do we know it isn't hiding a real regression?

And who verifies that an AI-generated test actually checks the correct behavior?

I believe the answer is a combination of AI assistance, deterministic checks, and human judgment.

AI can help us work faster, but we still need to understand what we're validating.

Final Thoughts

I don't think the future of QA is about writing more automated tests.

It's about finding better ways to prevent defects, identify risks earlier, and make sure we're building the right things.

AI can help us test faster. QA must help us test smarter.

I'm curious how other teams are approaching this.

Are you using AI mainly to generate more tests, or are you also changing how QA is involved in the development process?

Your LLM app regresses silently. Version-guard it before the next ship.

LXSAIHUB — Thu, 05 Nov 2026 08:00:17 +0000

LLM-powered features have a failure mode traditional software doesn't: they degrade without breaking. The output is still valid JSON, still 200 OK, just worse — and the regression hides until a user complains about something that "used to work."

I built TestPilot because "it worked in testing" is the most common and least useful sentence in LLM QA.

30-second takeaway

LLM regressions don't throw errors — they drift, and drift is invisible to normal tests.
The guard is an eval checklist tied to a version, run before every ship.
Catching the drift at commit time beats catching it in a support ticket.

The regression nobody's watching

A prompt change, a model swap, a system-message tweak — any of them can move behavior in a way unit tests never see. The app still runs. The answers just get softer, or wrong, or off-brand. By the time it's a pattern, it's shipped to everyone.

An eval tied to the version turns that invisible drift into a visible diff you can block on, the way you'd block a failing unit test.

What the tool does

Paste a prompt and its expected behavior; get an eval checklist and a version regression guard. Built for dev and QA teams shipping LLM-powered features. Open it and try — no signup.

TestPilot runs in the browser, no account, no key.

FAQ

What is TestPilot?

A tool that turns a prompt and expected behavior into an eval checklist and a version regression guard for LLM apps. No account, no key.

Does it replace my test suite?

No. It covers the behavior-drift your normal tests can't see. The two are teammates.

Will it call the model for me?

It helps you define what to check; the runs are yours to wire into your pipeline.

Author's Process

This product was built with AI coding tools. The design fight was against turning it into another agent — it stays a guard, not a chatbot.

nexus recorder

nexus studio — Thu, 05 Nov 2026 00:53:44 +0000

Writing a test should not start with a blank file.

Record what you do and Nexus Studio writes it as Playwright or Selenium code. No AI in the recorder. Or describe the test and your own AI drafts it, then seven checks without AI decide if it is safe to keep.

Nexus Studio by Verdict System

How to Build a QA Process from Scratch When Developers Own All the Testing

Mike Ralduxin — Wed, 04 Nov 2026 13:51:00 +0000

Your first week as the first QA hire usually looks the same regardless of the company. The bug tracker has 400 open tickets and nobody can tell you which ones are still real. Releases ship when the developer who wrote the feature feels done. Someone shows you a folder of automated tests that nobody has run in months. And leadership expects you to "add QA" — by which they usually mean "start writing automated tests immediately."

That last expectation is the one to push back on. After setting up QA functions from zero across dozens of engagements at DeviQA, the pattern I keep seeing is this: the first 90 days are mostly process archaeology and documentation, not test execution. Audit what exists, fix the bug tracker, write down what "working" means, then build smoke and regression suites and release gates. Automation comes last, because automation can only encode a process that already exists. Build it first and you encode the chaos.

Why developer-owned testing breaks down (and when it doesn't)

The failure mode is systemic, not a skills problem. Developers can test — they write unit tests, they check their own features. What's missing is ownership of the quality signal: nobody maintains the bug backlog, nobody defines what must pass before release, nobody tests the paths that cross feature boundaries.

BiznessApps, a DIY mobile-app platform for small businesses, is the cleanest example I've worked near. Before a QA function existed, developers were responsible for testing and executed it chaotically: the bug tracker was poorly set up with a messy board, bug reports were unstructured and uninformative, there was no test documentation, and releases were random, not preceded by thorough testing. The result was predictable — major and critical issues kept reaching production. None of that traces to bad developers. It traces to a testing process that no one owned.

The honest caveat: pre-product-market-fit, developer-owned testing can be a rational trade-off. If the product may not exist in six months, a dedicated QA process is overhead. It stops being rational the moment production incidents start consuming more engineering time than a QA function would cost — and at the macro level that cost is enormous: CISQ estimated poor software quality cost the US economy $2.41 trillion in 2022 (an older figure, but the most comprehensive one available). If you're being hired as the first QA engineer, someone has already decided the trade-off flipped.

Days 0–30 — audit before you change anything

The instinct on day one is to start fixing. Resist it. You can't prioritize what you haven't inventoried, and the audit is what turns "everything is broken" into a ranked list.

What a testing audit actually looks at

Four things, in order of how much they'll shape your next 60 days:

The release flow as it actually happens. Not the documented one — the real one. Who decides a build ships? What gets checked, by whom, in what environment? At BiznessApps the answer was "releases were random," and knowing that early meant release criteria became a 60–90 day priority rather than an afterthought.
The bug backlog's signal-to-noise ratio. Sample 50 tickets. How many are reproducible from the report alone? How many are already fixed, duplicated, or obsolete? The DeviQA team at BiznessApps began by retesting all legacy issues before changing anything else — tedious, but it's the only way to learn the product's actual state.
Environments. Can you get a stable test environment that isn't production? Can you get test data? If not, that's a blocker to escalate in week one, not month three.
Existing automation. Run whatever scripts exist and read what they assert. At RevenueHero, a lead-conversion platform, the audit found that the existing automation scripts were useless because they didn't check the required functionality. Green checkmarks, zero information. That's what automation built without a process underneath looks like — and it's why "just write tests" is the wrong day-one instruction.

One warning from experience: timebox the audit to two or three weeks. An audit that runs longer becomes a deliverable in itself, and you were not hired to produce a report.

Fix the bug tracker before writing a single test case

This is the highest-return early win, and it's unglamorous. An unusable board hides the product's real state from everyone — you can't build a regression suite around bugs you can't trust, and you can't argue for release gates when nobody agrees on how many open criticals exist.

The fix has three parts, all visible in the BiznessApps engagement: the Jira workflow was redesigned, obsolete entries were purged after retesting, and a bug-report template was introduced to formalize reporting — expected vs. actual behavior, reproduction steps, environment, severity. At Shops, a digital-products platform where QA had previously been one engineer verifying tickets, optimizing the Jira setup was likewise part of moving to a real process. This matters more than it sounds: per the State of Testing 2025 report, 77% of teams run their testing work through Jira, so the tracker effectively is the quality database. A minimal template that works:

Summary

One sentence: what is broken, where.

Environment

App version / branch, browser or device, test account.

Steps to reproduce

Expected / Actual

Expected: ...

Actual: ... (screenshot or log attached)

Severity

Blocker / Critical / Major / Minor — per the agreed definitions, not gut feel.

The trap here is political, not technical. The tracker is the developers' daily workflow, and redesigning it unilaterally breeds resistance that will follow you for a year. Propose the template, run it on your own reports for two weeks, show the difference in triage speed, then ask the team to adopt it.

Days 30–60 — test documentation and the first smoke suite

With a trusted tracker, write down what the product is supposed to do — starting from the critical paths, not from a coverage spreadsheet. Which flows, if broken, cost money or users within hours? Signup, checkout, the core workflow. Document those as structured test cases first; the smoke suite falls straight out of them.

Motion, a workflow-management platform, is the reference point for what this phase realistically produces. The starting state was absolute zero: no documented test cases, no smoke or regression testing, no automation, no CI integration. One full-stack DeviQA engineer built 80 structured, reusable test cases plus regular smoke and regression testing — and the whole engagement, including the automation that came later, ran five months. That's the honest scale for a first quarter with one person: dozens of solid cases covering critical paths, not thousands.

Don't chase full coverage on paper. Arklign, a dental-tech case-management platform, also started from nothing — no test cases, no release process, no QA process at all — and eventually reached 5,000+ web and 2,000+ mobile test cases at 95% coverage. "Eventually" is doing heavy lifting in that sentence: the engagement started in 2016. Writing five thousand cases in month two produces a documentation graveyard; writing forty that people actually run produces a smoke suite.

Days 60–90 — regression suite and release criteria

The regression suite grows outward from the smoke suite: same structure, wider coverage, organized by feature area so it can be split across people and later across automation threads. At Shops, that meant 500+ test cases covering more than 75% of the platform's functionality, with regression executable in one working day and smoke in under an hour — by a team of two, built over two years.

But the release criteria matter more than the suite's size. A regression suite answers "what do we check?"; release criteria answer "what must pass, and who signs off?" — and that second question is the one nobody has answered in a developer-owned setup. This is standard practice everywhere QA is established: 86% of testing teams have a say in release readiness per the State of Testing 2025 survey. A workable first version fits on one page: smoke suite passes 100%, zero open blockers or criticals, regression executed on release candidates, named sign-off.

BiznessApps shows where this leads. A clear release flow was established and QA joined the work on acceptance criteria; after the platform's rebuild, releases went out with zero major or critical production issues, support issues dropped by more than 50%, and testing surfaced over 1,700 issues including 300+ blockers along the way. Two honest qualifiers, though. That engagement ran six QA engineers over four years — your 90-day goal is a gated release, not that end state. And the support-ticket drop can't be attributed to QA changes alone; a lot changed at once, including the rebuild itself.

The automation roadmap is a 90-day output, not a day-one input

By day 90 you have what automation actually requires: documented critical paths, a smoke suite that's stable when run manually, and bug data showing where regressions cluster. The roadmap is now mechanical — automate the smoke suite first, then the regression cases with the highest execution frequency and the least UI churn.

Run in this order, automation compounds fast. At Motion, the same five-month engagement that started with zero documentation ended with 75+ automated end-to-end tests in Playwright and TypeScript, structured with the Page Object Model, running nightly and pre-release through GitHub Actions, covering 90% of critical features — and 40+ bugs logged, 7 of them critical. The nightly cadence is the part worth copying on day one of automation, and it's a few lines of CI config:

.github/workflows/nightly-e2e.yml

on:

schedule:

cron: "0 3 * * *" # nightly, before the team's day starts

workflow_dispatch: # and on demand before a release

jobs:

e2e:

runs-on: ubuntu-latest

steps:

uses: actions/checkout@v4
run: npm ci && npx playwright install --with-deps
run: npx playwright test --project=smoke

RevenueHero shows the same sequence at larger scale: after the useless legacy scripts were replaced with independent tests using API-driven data creation, the suite grew past 200 scripts and 70 API scenarios, coverage exceeded 90%, and full regression now runs in 1.5 hours. Arklign's automated smoke runs in 10 minutes across 900+ E2E and 100+ API tests, supporting 100+ web releases. One caution when you take these numbers into a planning meeting: they're ongoing-engagement figures — RevenueHero has been running since 2022, Arklign since 2016. They're the ceiling the first 90 days makes reachable, not a Q1 target.

There's also a broader reason the sequencing question matters right now. GitLab's 2025 Harris Poll survey of 3,266 DevSecOps professionals found teams losing about 7 hours per person per week to inefficient processes — automating on top of an inefficient process just executes the inefficiency faster.

What this won't fix

A QA process makes the product's state visible and releases deliberate. It does not fix poor requirements — you'll surface ambiguity earlier, but someone still has to resolve it. It doesn't conjure test environments or test data where none exist. It doesn't replace headcount math: every case above had dedicated QA staffing, from one engineer at Motion to six at BiznessApps. And it doesn't mean developers stop testing — unit tests remain theirs; what moves to QA is ownership of the integrated quality signal. One more disclosure, since this article leans on them: the case evidence here comes from vendor case studies, including our own. They're real engagements with real numbers, but they're not controlled comparisons, and I've tried to flag the limits of each as they came up.

Day 90, done right, looks modest on paper: a tracker the team trusts, documented critical paths, a smoke suite anyone can run, a gated release with named sign-off, and a ranked automation backlog. The 90%+ coverage and zero-critical-release numbers in the case studies above are multi-year outcomes. The first quarter doesn't deliver them — it's what makes them possible. Reach for this sequence when you're the first QA hire walking into developer-owned testing; skip the process-heavy version if the product is still searching for market fit and a failed release costs you nothing but a redeploy.