You have tested hundreds of features. You know the drill. Open the test case, write the preconditions, list the steps, fill in the expected result, run it, compare. Pass or fail. Move on.
Then someone hands you an AI feature. A chatbot. A "summarize this ticket" button. A search box that answers in full sentences instead of returning a list of links. You open your test case template, you get to the "expected result" field, and you stop.
Because what is the expected result? You run the same input twice and get two different answers. Both sound reasonable. Neither matches the other word for word. There is no string to assert against and no golden screenshot to diff. The one thing you have relied on your entire career, a known correct output, is gone.
Here is the reframe that matters. The problem is not that you forgot how to test. The problem is that the definition of "correct" changed, and nobody handed you a new one.
This guide is the new one. It is a practical playbook for testing AI features manually, with no code required. It covers why AI features break your usual playbook, how to write acceptance criteria when there is no single correct answer, how to probe for hallucinations and bias by hand, how to document a bug that refuses to reproduce, and why manual testing of AI systems is quickly becoming the most valuable thing you can put on your resume.
Almost everything written about testing AI assumes you write code. Katalon has covered that engineering angle in depth, from the new challenges AI brings to software testing to why AI-generated code needs AI-powered testing. This is the manual tester's counterpart: no pipeline, no scripts, just you, the feature, and your judgment. It is also the most urgent skills gap in manual testing right now, and the rest of this guide is about closing it.
THE SHORT ANSWER
How to test an AI feature manually, at a glance:
• Probe it with questions you already know the answer to.
• Ask it something it cannot possibly know, and see if it admits that.
• Run the same input five to ten times and read the spread, not just one result.
• Push the edges: empty input, gibberish, contradictory instructions.
• Check every citation the feature gives you.
• Swap one irrelevant attribute, a name or a postal code, and compare outcomes for bias.
The rest of this guide walks through each of these in depth, plus how to define acceptance criteria and document a bug when there is no single correct answer.
Why AI Features Break Your Usual Playbook
Traditional testing rests on one assumption: for a given input, there is a correct output you can define in advance. Click "add to cart," the cart count goes to 1. Submit the form with a blank email, you get the "email required" error. The oracle, the source of truth that tells you what should happen, is known before you run the test.
AI features remove that assumption. The same prompt can produce different wording every time. "Correct" becomes a range of acceptable answers rather than a single value. This is non-deterministic testing, and it breaks the muscle memory of every tester who learned to verify against an exact expected result.
It also introduces failure modes you have never had to look for:
- Hallucinations: The model states something false with complete confidence. An invented refund policy, a citation to a document that does not exist, a product spec that was never true.
- Bias: The feature treats equivalent users differently based on names, gender, age, or location, the kind of differential behavior that turns a feature into a legal and ethical problem.
- Ungrounded confidence: The output sounds authoritative regardless of whether it is right. Tone is no longer a clue to correctness.
- Soft failures: Traditional bugs are loud: a crash, a stack trace, a 500. AI bugs are quiet. The answer looks fine, reads fine, and is wrong. Nobody catches it because nothing looked broken.
The scale of the problem is not anecdotal. Stanford's 2026 AI Index Report benchmarked hallucination rates across 26 leading models and found error rates ranging from 22% to 94%, with some models degrading sharply the moment a false claim was framed as something the user already believed. That is the feature you are being asked to test: a system that can be right most of the time and still confidently wrong in a way no crash log will ever surface. Testing AI features is no longer an edge case bolted onto your normal work. It is its own competency.
So the question is not whether you can test AI applications. You can. The question is what replaces "expected result" when there is no single right answer. That starts with acceptance criteria.
Define Acceptance Criteria When There Is No Single Correct Answer
You cannot test what you cannot define. When the output is non-deterministic, you stop defining a single answer and start defining the boundaries of an acceptable one. The shift is from "the output equals X" to "the output satisfies these properties."
Work with your product owner to pin down those properties before you test. For most AI features, acceptable behavior comes down to a handful of dimensions:
| Dimension | The question it answers | Example criterion |
|---|---|---|
| Grounding | Is the answer based on real, provided data? | Every factual claim traces to the source document; nothing is invented |
| Boundaries | Does it know what it does not know? | When the answer is not in scope, it says so instead of guessing |
| Guardrails | What must it never do? | Never invent a policy, price, or legal commitment |
| Relevance | Does it actually answer the question? | Response addresses the user's intent, not a tangent |
| Format | Is the output usable? | Returns valid JSON, stays under the length limit, or uses the required tone |
| Safety and fairness | Does it treat people equally? | Equivalent inputs produce equivalent outcomes across demographics |
Notice what these criteria have in common. None of them require a single correct string. They define a space of acceptable outputs and, just as importantly, a space of unacceptable ones. "Must never invent a refund policy" is testable even when "the perfect answer" is not.
You do not have to invent this framework from nothing. OWASP's Top 10 for LLM Applications names several of these same failure modes as top-tier risks: misinformation covers ungrounded, hallucinated output, excessive agency covers a system acting or promising beyond its actual authority, and improper output handling covers formats and structures that break downstream systems. Borrowing that vocabulary when you write acceptance criteria gives your product owner a named, external standard to anchor against instead of a list you made up on the spot.
This is also where your domain knowledge becomes the product. An AI feature does not know your business rules, your compliance constraints, or the difference between a confident-sounding answer and a true one. You do. The acceptance criteria you write are where that knowledge gets encoded into something testable.
A Manual Playbook for Testing Hallucinations
Hallucinations are the failure mode that scares teams the most, because the output looks exactly like a correct answer. You do not need code to catch them. You need a method. Here are the techniques that work by hand.
Probe with questions you already know the answer to
Feed the feature inputs where you know the ground truth cold. Ask it about a document you wrote, a policy you can recite, a record you can verify. If it gets a known answer wrong, you have found a hallucination without any tooling at all. Start here every time.
Ask what it cannot possibly know
Give it questions that have no answer in its data: a customer who does not exist, a feature you never shipped, a date in the future. A well-built feature says "I do not have that information." A hallucinating one invents a confident, detailed, completely fictional response. This single test separates features that know their boundaries from features that do not.
Run the same input several times
Because the output is non-deterministic, one run tells you almost nothing. Run the same prompt five or ten times and read the spread. Consistent and correct is good. Consistent and wrong is a clear bug. Sometimes right and sometimes wrong is the most important finding of all, because it will never show up if you test once. Record how often the failure appears. "Hallucinated 3 of 10 runs" is real testing data.
Push the edges
Empty input. Gibberish. A 10,000-character wall of text. Two contradictory instructions in the same prompt. A question in a language the feature was not built for. Edge inputs are where grounding breaks down and the model starts filling gaps with invention. This is the same instinct you already have as a tester, pointed at a new target.
Check the receipts
If the feature cites sources, open them. Hallucinated citations are everywhere: real-looking references to documents that do not exist, or links to real documents that do not say what the feature claims. A citation is not evidence until you have read it. Testing LLM outputs means verifying the support, not just the answer.
A Manual Playbook for Testing Bias
AI bias testing is not a "nice to have" check. When an AI feature influences hiring, lending, insurance, or any decision about a person, biased behavior is both an ethical failure and, increasingly, a legal one. The EU AI Act classifies systems used in areas like employment and credit scoring as "high-risk," and high-risk systems carry obligations that include human oversight and technical documentation of how the system behaves. Someone has to generate the evidence that the system treats people fairly. That someone is a tester.
The core technique is equivalence swapping. Build pairs of inputs that are identical in everything that should matter and different only in something that should not:
- The same resume with the name "Michael" and then "Aisha."
- The same loan profile in two postal codes.
- The same support question phrased in fluent English and then in broken English.
- The same medical query for a man and for a woman.
Run both. Compare the outcomes. If the feature recommends, scores, or responds differently when only the irrelevant attribute changed, you have found bias, and you have the side-by-side evidence to prove it. Vary one attribute at a time so the cause is unambiguous, and cover the attributes that matter for your product and your regulators.
This is slow, deliberate, human work. It depends on imagining how real people differ and what fairness means in your specific domain. No script writes these test cases for you, because no script knows which differences are supposed to be invisible.
How to Document a Bug When There Is No Single Correct Answer
You found something. Now you have to write it up, and your usual bug template fights you the whole way. "Steps to reproduce" assumes it reproduces. "Expected vs. actual" assumes a known expected. Neither holds for a non-deterministic feature. So adapt the report.
Capture more context than you are used to:
- The exact input, word for word, including any conversation or documents that preceded it.
- The full output, copied verbatim, not paraphrased. Screenshot it too.
- Frequency, not reproducibility. "Occurred in 3 of 10 identical runs" replaces "always reproducible." It tells the team this is intermittent and roughly how often.
- The environment that shapes AI behavior: model name and version, date and time, and any settings you can see, such as the temperature or which knowledge source was connected.
- Why it is wrong, against the criteria. This is the key move. Do not write "the answer was bad." Write "violated the grounding criterion: invented a 30-day refund policy; our actual policy is 14 days and is the only one in the source document." You are mapping the failure to an agreed rule, not to your opinion.
- Classification: hallucination, bias, grounding failure, formatting, or missing refusal. A label helps the team route and trend the issue.
Here is the difference in practice:
The second report is something an engineer can act on, a compliance lead can file, and a product owner can prioritize. It also happens to be exactly the kind of documented, traceable evidence the EU AI Act expects teams to keep for high-risk systems. Good bug hygiene and regulatory readiness turn out to be the same habit.
Why This Is the Manual Testing Skill AI Cannot Touch
There is a quiet anxiety running through manual testing right now. If AI can generate test cases and even run some of them, what is left for the human? It is a fair question, and testing AI features is the clearest answer to it.
Think about what every technique in this guide actually required. Deciding whether a non-deterministic output is acceptable. Knowing the real refund policy. Imagining how a frustrated or unusual user behaves. Recognizing that two answers were treated differently when they should not have been. Judging a confident-sounding response and saying "that is wrong." None of that is pattern matching. All of it is judgment, domain knowledge, and an understanding of real people.
An AI system cannot be its own trustworthy oracle. It cannot reliably decide whether its own output is correct, fair, or safe, for the same reason it produced the questionable output in the first place. Something outside the system has to make that call.
So the demand here is not hype. It is structural. OWASP now treats misinformation and excessive agency as top-tier risks in their own right, ISTQB has turned AI-system testing into a named, certifiable specialism, and Stanford's own benchmark data shows these failure modes are not rare edge cases. A manual tester who can test AI applications, probe for hallucinations, design bias checks, and produce defensible documentation is not being automated away. They are becoming the person the team cannot ship without.
The testers who thrive will not be the ones who wrote the most test cases. They will be the ones who can look at a system no script fully understands and say, with evidence, whether it is safe to trust.
Common Mistakes When Testing AI Features
Trusting a single run. One good answer proves nothing when the output is non-deterministic. If you did not run it several times, you did not test it.
Mistaking confidence for correctness: AI features state wrong answers in the same authoritative tone as right ones. Tone is not a signal. Verify the substance every time.
Only testing the happy path: The interesting failures live at the edges and outside the model's knowledge. If you never asked it something it cannot know, you never tested for hallucination.
Defining acceptance as an exact match: Holding a non-deterministic feature to a single expected string guarantees noise. Define properties and boundaries instead.
Skipping the environment details: A bug report with no model version, date, or settings is almost impossible to investigate later. AI behavior changes with all three.
Testing one persona, one language, one region: Bias hides in the cases you did not vary. If every test used the same kind of user, you have not checked for fairness.
How Katalon True Platform Supports Manual Testing of AI Features
Testing AI features by hand still needs structure, traceability, and a place to keep the evidence. That is the part a platform should carry so you can focus on judgment.
Katalon True Platform is built around a simple division of labor: AI drafts, you decide. A few ways it supports the manual testing of AI systems:
- Structured manual testing and test management. Store your AI feature test cases, including property-based acceptance criteria and the "what it must never do" guardrails, in one place, and attach transcripts and screenshots as evidence for the runs that matter.
- Requirement traceability by default. Each test links back to the requirement it verifies, so the documented relationship between "what we promised" and "what we tested" is built in, the kind of trail high-risk AI systems are expected to maintain.
- AI assistance with a human gate. The Requirement Analyzer scores a requirement for testability and surfaces ambiguity, which is exactly where fuzzy AI acceptance criteria tend to hide. The Test Generation Agent can draft a first set of cases, and then you do the work only a human can: harden them against the review checklist that catches what AI misses, and decide what "acceptable" means.
The platform treats AI as a drafting and execution aid and keeps the human as the judge of quality, the same division of labor behind Katalon's broader AI testing governance framework.
Key Takeaways
- AI features break traditional testing because the output is non-deterministic: the same input can produce many different, equally valid answers, so "expected result" stops working.
- Replace the single expected output with acceptance criteria defined as properties and boundaries: grounding, knowing its limits, hard guardrails, relevance, format, and fairness. OWASP's Top 10 for LLM Applications gives you a named vocabulary for several of these.
- You can test for hallucinations by hand: probe with known answers, ask what the system cannot know, run the same input many times, push the edges, and verify every citation.
- Test for bias with equivalence swapping: change only the attribute that should not matter, compare the outcomes, and keep the side-by-side evidence.
- Document non-deterministic bugs by frequency, not reproducibility, and explain why the output is wrong against an agreed criterion. Capture model, version, date, and settings.
- This is judgment work AI cannot do for itself. Independent benchmarks put hallucination rates as high as 94 percent on some models, and both OWASP and ISTQB have turned AI-system testing and its risks into named, structured disciplines.
- Manual testers who specialize in testing AI features are becoming more valuable, not less. The skill is in structural demand, not on the way out.
You do not need to write code to test AI features well. You need a method, the judgment to apply it, and a place to keep the evidence. Katalon True Platform gives manual testers structured test management, automatic traceability, and AI assistance that always defers to a human decision. AI drafts. You decide what ships.
References
- Stanford HAI. "The 2026 AI Index Report: Responsible AI." Stanford Institute for Human-Centered Artificial Intelligence, 2026. hai.stanford.edu
- OWASP. "OWASP Top 10 for Large Language Model Applications (2025)." OWASP Gen AI Security Project. genai.owasp.org
- ISTQB. "ISTQB Releases Certified Tester AI Testing (CT-AI) Syllabus Version 2.0." International Software Testing Qualifications Board, April 2026. istqb.org
- European Commission. "Guidelines for providers and deployers of high-risk AI systems." Shaping Europe's Digital Future, 2026. digital-strategy.ec.europa.eu
- EU Artificial Intelligence Act. "Article 60: Testing of High-Risk AI Systems in Real World Conditions." artificialintelligenceact.eu
Top comments (0)