Hey dev.to!
Ilya here again. In my Previous article, I explained my reasons for no longer generating code, but instead to parse plain English into steps that I store in a database.
I did this because (as the only QA on my team) I was tired of wasting time writing and maintaining test code. My hypothesis was that "describing" a test case should be many times faster than "coding" it.
That sounds nice in theory. But what about in practice?
I decided to run an honest experiment. I didn't pick simple "login-logout" examples. I took three real-world test cases from my backlog—from simple to complex—and I timed them.
Important: I wasn't timing the test execution (that's fast, 1-2 minutes). I was timing the creation—how long it took the AI to understand my description and generate all the steps.
Here's what happened.
Test 1: The "Simple but Annoying" One (EULA Popup)
Goal: Test the EULA (End User License Agreement) popup that appears after login.
My Description: login on the EULA popup click accept
Result:
Generation Time: 00:58 (58 seconds)
Steps: 4 (3 of which are my standard UI login)
My Comment: Less than a minute. Honestly, it would have taken me longer just to find the correct xpath or data-testid for that "Accept" button than it took the AI to generate the entire test. That's a quick win.
Test 2: The "Sweet Spot" (E2E User Creation)
This is a classic E2E test that checks a full feature flow.
Goal: Create a new user with an "Administrator" role, log out, and then log in as that new user to verify it was created.
My Description: login; Navigate to users page; Create a new user; Fill all required fields, select (email: damin.example.123@test.com, name: damin, password: 123, Role: Administrator); Logout; Login with new user (email: damin.example.123@test.com, password: 123)
Result:
Generation Time: 06:38 (6 minutes 38 seconds)
Steps: 20 (3 of which are my standard UI login)
My Comment: This is it. A 20-step E2E test created in six and a half minutes. This is a complete flow: login, navigation, filling a form (including a dropdown selection!), logout, and logging back in with the new data.
How long would it have taken me to code this by hand? Finding all the selectors, writing the waits, and debugging it? An hour? Maybe two? Instead, I just described the flow like I was writing a task for a junior tester.
Test 3: The "Complex" One (Verify Download)
Okay, but what about a really complex scenario with multiple verifications?
Goal: Log in, go to the templates page, download one, and then verify it appears in the downloaded list.
My Description: login to website; Go to Templates -> Download Templates; Download an awareness template; select a template to download; click download template; verify if new template is in the downloaded template list
Result:
Generation Time: 23:38 (23 minutes 38 seconds)
Steps: 39 (again 3 of which are my standard UI login)
My Comment: Okay, almost 24 minutes. That's not "instant." The AI had to "think" for a long time to correctly link all these steps and find all the elements.
But let's be honest. This is a 39-step test. This is a serious, complex scenario. How long would it take to code 39 steps, including the logic to verify a file appeared in a list? Several hours, maybe half a day.
This is the whole point of Debuggo for me.
What's the takeaway?
This tool isn't a "magic button" that solves everything in a second. The AI (especially on complex tasks) needs time to "think."
But it fundamentally changes my time economy. It turns hours of coding into minutes of describing.
For me, as the only QA, that's the difference between "I have no time for automation" and "I have 20 tests I created today."
These are my results. Now, I need yours. I can't be sure this works until I see how Debuggo handles your websites.
Please, try it and time it for yourself.
Sign up for the beta at https://debuggo.app.
Top comments (0)