Lately, I’ve been thinking about something that sounds like a good problem to have: AI can help us create a lot more tests, but does having more tests actually mean we have better coverage?
There are already some interesting approaches. Katalon can generate test cases from requirements, ACCELQ looks at more autonomous test generation, LambdaTest’s KaneAI works from natural-language scenarios, and Functionize can turn test instructions into executable tests. Each approach makes creating tests easier, but I think the harder part starts after the generation.
While exploring X360 AI Tech, I started looking at this from another angle. Instead of asking how many tests AI can generate, I’m more interested in whether those tests actually cover the important parts of the requirement - the main user flow, edge cases, business rules, and things that could easily be missed.
Because AI can generate 20 tests where maybe only 5 are really useful. And it can also miss one important scenario while giving us the feeling that we have good coverage.
I’m starting to think that the number of AI-generated tests is probably not the best measure of coverage. I’d rather have fewer tests that cover the right risks than a huge suite full of similar scenarios.
For me, that’s where QA still matters. The question isn’t just “How many tests did AI create?” but “What did those tests actually cover?”
Top comments (0)