DEV Community: Jamilur Rahman

A 500-Line Flutter Login Test Became One Promt

Jamilur Rahman — Wed, 05 Aug 2026 20:44:14 +0000

Lets start with a bit of back story.

I am a full stack developer. Developer being the keyword here, not a QA developer.

But in my current role, I was recently asked to come up with a testing suite for the web application and the Flutter app I was managing and maintaining.

At that time, I didn’t have anything better to do and thought this would be a fun little project to work on for a couple of weeks.

Boy o boy, I was wrong.

People in QA are so opinionated.

Everyone has their preferred framework, structure, naming convention, abstraction, folder structure and a very strong opinion about why your approach is wrong.

Starting with the industry best practices

I started by trying to follow the trends and best practices used in the industry.

Page Object Models, reusable helpers, proper assertions and all the usual bits and bobs.

For the web application, which was built with React, I chose Playwright.

For the Flutter app, I went with integration_test.

Sounded simple enough.

The login test that took three hours

The first test I tried to write was a simple login flow.

Open the application
Enter the username and password
Press the login button
Wait for the dashboard

Easy, right?

It took me ages.

And by ages, I mean roughly three hours just to get the web test to pass reliably.

The actual Playwright test ended up being around 300 lines once I included the boilerplate, setup, selectors, assertions, waits, Page Object Model structure and everything else needed around the actual journey.

Then came the Flutter app.

That one was worse.

The app has its own custom way of starting different flavors, and both the web application and Flutter app are white-labelled products.

That means there are a lot of variations to cover.

Different branding, configurations, screens and sometimes slightly different user journeys.

Before I could even test the login flow, I needed a pile of setup code just to launch the correct version of the app.

The Flutter test eventually went beyond 500 lines, including the boilerplate.

I couldn’t even get it to run on the first day.

On the next day, with a fresh cup of coffee and Claude sitting beside me like an unpaid junior developer, I finally managed to get somewhere.

It ran.

But it was still flaky.

Fixed waits are a trap

The APIs did not always respond at the same speed.

Sometimes the dashboard loaded quickly.

Sometimes it took a few extra seconds.

Sometimes a loading indicator appeared.

Sometimes it disappeared almost instantly.

So fixed waits were unreliable.

Wait too little and the test fails.

Wait too long and the whole suite becomes painfully slow.

Then you add more conditions, more retries, more pumpAndSettle, more element checks and more code around what should be an extremely simple user journey.

At one point I stopped and thought:

Why am I explaining every single technical action when the actual requirement is obvious?

The requirement was simply:

The user should be able to launch the app, log in and reach the dashboard.

So I decided to break down the same journey as a prompt and let Claude try to execute it.

The prompt was something like this:


Launch the app and wait for loading to finish. Tap login, fill in the username and password, submit the form and verify the dashboard appears.

That was basically it.

Less than 120 characters of actual testing intent, depending on how you phrase it.

And on the first try, it got it right.

I was starstruck.

What actually changed

Obviously the prompt does not magically remove all the complexity.

Something still needs to:

Launch the app
Inspect the interface
Find the relevant elements
Interact with them
Wait for the correct states
Decide whether the journey passed or failed

But that responsibility no longer had to sit inside hundreds of lines of test code written and maintained by me.

With Playwright or Flutter integration tests, I had to explain how every action should happen.

With the prompt, I only had to explain what the user was trying to achieve.

That was the aha moment.

From one prompt to full user journeys

From there, I kept adding more prompts for different user journeys.

Create a beneficiary.

Start a transaction.

Complete multi-step forms.

Check that records appeared in the correct list.

The only scripts I wanted to maintain were the ones needed to launch the Flutter app or open the web application.

The rest of the journey could be described in natural language.

The next journey was much easier to add.

Then the next one.

And the next one after that.

It was not perfect

I don’t want to make this sound like one prompt suddenly solved every testing problem.

It didn’t.

Sometimes the instructions were too vague.

Sometimes the application entered a state the agent could not recover from.

Sometimes a site blocked automation completely.

And traditional coded tests are still better when you need complete determinism and precise control over every interaction.

Playwright is also faster and more predictable for stable flows where you already know exactly what should happen.

But for business-level user journeys, this felt completely different.

I was no longer spending most of my time writing selectors, waits and boilerplate.

I was describing what the user should be able to do.

That small moment became CueTest

That small aha moment eventually gave birth to CueTest.

CueTest lets you describe end-to-end tests in natural language and watch the browser execute them.

The Android testing side is not live yet, but the web testing platform is available now.

I am still figuring out how far this approach can go, where it breaks and where traditional frameworks still make more sense.

So please don’t expect a polished marketing answer from me.

Try it.

Break it.

Give it a weird user journey.

And leave me your honest feedback.

I would especially love to hear from people maintaining Playwright, Selenium, Cypress or Flutter integration test suites.

Do you think agentic testing will eventually replace most traditional end-to-end scripts, or will it mainly sit beside them?

Agentic E2E Testing Without the AI Hype: What It Actually Means

Jamilur Rahman — Wed, 05 Aug 2026 18:00:55 +0000

I keep seeing two phrases buzzing around my feed:

Agentic testing and AI-powered testing.

They are often used as though they mean the same thing. They don’t.

So let’s break this down first.

What does AI-powered testing actually mean?

“AI-powered testing” can mean many different things.

It might mean using AI to generate tests for frameworks such as Playwright, Selenium, Cypress, or whatever testing framework your QA team refuses to stop talking about.

The QA team at my office loves Selenium and keeps pushing for it.

Insider news: I am not letting that happen while I am still here.

Jokes aside, AI-generated test code is probably the most common version of AI-powered testing today.

You give Claude, Codex, Cursor, or another coding assistant a description of the feature, and it produces a test script for you.

The result is still a traditional code-based test.

AI simply helped write it.

AI-powered testing can also mean adding intelligence around an existing test suite.

For example, you may already have Playwright or Selenium tests. When one of them fails, AI can help determine whether:

The application is genuinely broken
The test is outdated
A selector changed
The failure was caused by timing or flaky behaviour
The user journey itself has changed

AI can also analyse logs, screenshots, browser activity, and failure messages to explain what probably happened.

Another version is AI-assisted recovery.

A test fails at a particular step, but instead of stopping immediately, the system tries to understand the page and find another valid way to continue.

This is commonly called self-healing test automation.

A few years ago, doing this reliably was extremely difficult. Even now, “self-healing” is often oversold, but modern browser agents and language models have made limited recovery much more practical.

So AI-powered testing can mean:

Generating test code
Analysing test failures
Updating outdated tests
Recovering from broken selectors
Explaining whether a failure came from the product or the test

Agentic testing is slightly different.

What is agentic E2E testing?

With agentic testing, you are not necessarily writing the test implementation yourself.

You describe the user journey, expected outcome, or task in natural language.

The agent then uses the tools available to it to understand the application and execute the journey.

For a web application, those tools might allow the agent to:

Inspect the current page
Find interactive elements
Click buttons
Fill forms
Navigate between screens
Read visible content
Check whether an expected result appeared
Recover when a small UI change blocks the original path

For Android, iOS, desktop software, or other weird things you somehow ended up having to test, the available tools would be different.

The main idea remains the same:

You describe the goal.

The agent decides how to complete it.

This workflow does not make much sense for unit testing. Unit tests require precise inputs, outputs, assertions, and deterministic behaviour.

But it can be extremely useful for end-to-end testing.

E2E tests are about user journeys.

Can the user sign in?

Can they create an account?

Can they add a product to the cart?

Can they complete checkout?

Can they create a transaction?

Can they reach the expected confirmation screen?

These are exactly the kinds of tasks that a browser agent can understand and attempt inside a real application.

The problem I had with traditional test automation

I am a developer, not a dedicated test automation engineer.

I can write tests, but I do not always know the best way to structure a serious E2E testing suite from day one.

This is probably common in smaller companies.

You may not have a full QA automation team. Management still wants important workflows covered, so the responsibility lands on whichever developer looked least busy during the meeting.

That developer may be you.

It has definitely been me.

The usual solution is straightforward.

You open Claude, Codex, Cursor, or another coding assistant and ask it to generate a Playwright or Selenium test.

The generated test looks reasonable.

It runs.

You get a green check mark.

You call it a day.

Then a few days later, somebody pushes a UI change.

A new button appears.

A label is renamed.

The page structure changes.

A table starts overflowing on smaller screens.

The main test might still pass because the original journey is technically working.

But the new button is not covered.

The overflowing table is not detected.

Or the test fails because a selector changed, and now you are asked to figure out whether the application broke or the test simply became outdated.

Adding coverage for the button is easy enough.

But what about the table overflow?

How exactly should I script that?

How do I test layout issues across multiple screen sizes?

How do I distinguish a harmless DOM change from a genuine user-facing regression?

With my limited testing knowledge, my first instinct would be to ask Claude again.

Then I would ask it to fix the first generated test.

Then fix the fix.

Then add another helper.

Then update the selector strategy.

Before long, the test suite becomes an AI-generated soup that technically belongs to me but is no longer something I confidently understand or want to maintain.

That was the problem I kept running into.

My first experiment with agentic testing

I started wondering whether the process could be simplified.

Instead of asking AI to keep generating and repairing test code, what would happen if the AI agent directly executed the user journey?

That led me toward agentic E2E testing.

The idea looked genuinely exciting.

Rather than writing every click, selector, wait condition, and fallback path, I could describe the flow and let an agent operate the browser.

So I started building a small internal testing suite around the idea.

Then I kept building.

Nobody stopped me and asked:

Where exactly are you going with this?

Honestly, I did not know either.

At first, I only wanted a better way to test my own applications.

But when the initial version started working, I could watch the agent interact with the application in real time.

It would open the browser, inspect the page, navigate through the user journey, fill forms, make decisions, and report what happened.

I could see the web application being tested live instead of staring at a terminal and waiting for a pass or fail message.

More importantly, the final result included useful context.

Not just:

Test failed.

But information about where the agent got blocked, what it expected to find, what it actually found, and what might have caused the problem.

That small experiment slowly turned into an MVP for an agentic E2E testing platform.

And eventually, CueTest came out of it.

What CueTest does

CueTest is an agentic end-to-end testing platform for web applications.

You describe the user journey using natural language.

CueTest runs that journey against your staging or deployed application using an agent inside a real hosted browser.

While the test is running, you can watch the browser session and follow the live execution logs.

The agent can:

Navigate through the application
Interact with forms and UI elements
Verify expected outcomes
Handle limited UI changes
Attempt bounded recovery when an action fails
Stop when the application is genuinely blocked
Analyse failures and provide useful context

The important difference is that the test is based on the intended user journey rather than a fixed sequence of selectors.

For example, suppose your original instruction is:

Sign in, open the beneficiaries page, create a new beneficiary, and verify that it appears in the list.

Later, the UI changes.

The button moves.

The wording changes from “Add beneficiary” to “Create beneficiary.”

The form layout is updated.

A traditional test may fail because the implementation changed, even though the user journey stayed exactly the same.

An agentic test may still complete the journey because it is trying to achieve the goal rather than blindly replaying one exact sequence.

That does not mean the agent will magically survive every redesign.

It will not.

Agentic testing is not magic, and it is definitely not perfectly deterministic.

But when the journey remains the same and the UI changes slightly, the original natural-language test can often continue working without being rewritten.

When the journey itself changes, you update the description.

Instead of refactoring a large test script, you explain the new flow.

Agentic testing is not replacing Playwright

I do not think agentic testing will completely replace Playwright, Selenium, Cypress, or other code-based testing frameworks.

Code-based tests still provide more precise control.

They are better when you need:

Strict assertions
Fully deterministic execution
Complex fixtures
Custom setup and teardown logic
Detailed control over network requests
Deep integration with engineering workflows
Extremely predictable regression coverage

Agentic testing offers a different trade-off.

It can make tests faster to create, easier to understand, and more adaptable to smaller UI changes.

It can also make E2E testing more accessible to developers, manual QA testers, product managers, and small teams that do not have the resources to build and maintain a large automation framework.

The strongest approach will probably be a combination of both.

Use code-based tests for critical workflows that require maximum predictability.

Use agentic tests for broader user journeys, rapidly changing features, exploratory coverage, and scenarios where writing and maintaining a full scripted test would be too expensive.

The harder problem is trust

Getting an AI agent to click buttons is not the hardest part.

The harder part is building a testing system that teams can understand and trust.

The agent needs boundaries.

It should not keep retrying forever.

It should not invent credentials.

It should not randomly explore unrelated pages.

It should not mark a test as passed simply because it managed to click through several screens.

It needs clear objectives, visible execution, limited recovery, meaningful stopping conditions, and useful failure analysis.

That is the part I am continuing to work on with CueTest.

The real question is not whether an AI agent can operate a browser.

It clearly can.

The more interesting question is:

How much uncertainty are teams willing to accept in exchange for faster test creation, broader coverage, and lower maintenance?

I would genuinely like to hear how other developers and QA engineers see this.

Which parts of your E2E test suite would you trust to an agent?

And which parts would you always keep code-based?

I am building CueTest, an agentic E2E testing platform for creating and running browser tests from natural-language user journeys.

You can check it out at cuetest.dev.

This article is based on my own experience building CueTest. I used AI to help organize and polish the writing, but the technical decisions, product experience, opinions, and questionable Selenium jokes are mine.