DEV Community

Patrick Bertsch
Patrick Bertsch

Posted on

I got frustrated with Flutter E2E testing… so I built my own tool

#flutter #testing #opensource #mobile

If you've done end-to-end (E2E) testing in Flutter, you probably know the feeling:

  • Tests get slow as they grow
  • Debugging is painful
  • Writing tests feels heavier than it should

I hit those limits pretty quickly with integration_test. I tried other options (including Patrol), but I still wanted something that felt faster and simpler — something where writing tests didn't feel like a chore.

So I started building my own Flutter E2E testing framework:

👉 FlutterProbe — open-source, BSL 1.1 licensed

The goal

I wasn't trying to reinvent testing — just make it feel better to use.

What I wanted:

  • ⚡ Fast feedback — closer to unit test speed, not minutes-long integration runs
  • ✍️ Tests anyone can read — not just the developer who wrote them
  • 🧪 Less flakiness — no more pumpAndSettle timeouts
  • 🧠 A simple mental model — describe what the user does, not how the framework works

What it looks like

Instead of Dart boilerplate with WidgetTester and find.byKey, you write plain English:

test "user can sign in with valid credentials"
  open the app
  tap "Sign In"
  type "test@example.com" into "Email"
  type "password123" into "Password"
  tap "Continue"
  see "Dashboard"
Enter fullscreen mode Exit fullscreen mode

That's actual ProbeScript — the test language FlutterProbe uses.
No Dart imports, no pumpAndSettle, no find.byType. Just behavior.

Under the hood, FlutterProbe connects to a Dart agent running inside your app via WebSocket. The agent walks the live widget tree directly — no UI automation layer, no WebDriver.

👉 This is how it achieves sub-50ms command round-trips.

Why not just use integration_test?

integration_test is Flutter's official E2E option, and it's solid for:

  • Official support and ecosystem integration
  • Basic smoke tests and simple flows
  • Teams already deep in the Dart test ecosystem

But for me, it starts to hurt when:

  • Tests grow in size — the boilerplate compounds fast
  • You need faster iteration — full rebuilds on every change
  • You want cleaner test code — tester.pumpAndSettle() everywhere
  • You need reporting — no built-in JUnit/JSON/HTML reports
  • You want CI/CD parallelism — no sharding support out of the box

FlutterProbe addresses each of these:

  • Human-readable syntax
  • Sub-50ms execution
  • Built-in reporters
  • --shard for CI matrix jobs
  • --parallel for multi-device runs

👉 Full comparison: FlutterProbe vs integration_test

And Patrol?

Patrol solves a lot — especially around native interactions (permission dialogs, system alerts, notifications). It's a serious tool and a real step up from vanilla integration_test.

FlutterProbe is trying something slightly different:

  • Plain English syntax — readable by QA, PMs, and developers
  • Direct widget-tree access — no Appium, no native automation layer
  • Speed — sub-50ms per command
  • Migration — supports Maestro, Gherkin, Robot Framework, Detox, and Appium
  • Cloud device farms — BrowserStack, Sauce Labs, AWS Device Farm, Firebase Test Lab, LambdaTest

If you need native OS interactions, Patrol is the better choice.
If you want speed, readability, and CI/CD-first design, FlutterProbe is worth a look.

👉 Full comparison: FlutterProbe vs Patrol

What's different so far

Feature FlutterProbe integration_test Patrol
Test syntax Plain English (ProbeScript) Dart Dart + custom finders
Execution speed <50ms per command ~200–500ms ~100–300ms
CI/CD sharding Built-in (--shard) Manual Manual
Parallel devices --parallel No No
Cloud device farms 5 providers No No
Visual regression Built-in No No
Test recording Yes No No
Migration tools 7 formats No No
Reports (HTML/JSON/JUnit) Built-in Manual Manual

Plus:

  • Self-healing selectors
  • Data-driven tests (CSV support)
  • Random data generators
  • Clipboard, GPS, permission commands
  • before all / after all hooks
  • HTTP mocking
  • VS Code extension with CodeLens + IntelliSense

When this might help you

If you:

  • Feel limited by integration_test boilerplate and speed
  • Want faster E2E feedback loops in CI/CD
  • Prefer test files that non-developers can read
  • Need multi-device or cloud testing
  • Are migrating from Maestro, Detox, or Appium

Still early — I'd love feedback

I'm actively working on this and would love input from people doing Flutter testing in production:

  • What's your biggest pain point with Flutter E2E testing today?
  • What tools are you using, and what's missing?
  • What would make E2E testing actually enjoyable?

Drop a comment — I read every one.

Learn More

Try it out

👉 GitHub: https://github.com/AlphaWaveSystems/flutter-probe
👉 Docs: https://flutterprobe.dev/

If it looks useful, a ⭐ helps a lot 🙌

Top comments (0)