DEV Community

Taweechai Maklay
Taweechai Maklay

Posted on

Flutter app automate testing with Appium Flutter Driver

As tech person whom adopted flutter in tech portfolio (got much friction around this). The most question I've got is testing automation engineer did not know dart and appium not officialy support flutter, how can they write test for it, how can they do automate test with it? So I will share context about how we can test flutter application with appium-flutter-driver with a practical example.

Project setup

  • Since Flutter did not use native conponents, it's mean you need to define keys in all of your widget to made your widgets visible to appium-flutter-driver (also for normal flutter testing).

Be aware! your testing apk need to be "debug" or "profiling" package.

Example

Text(
  'Welcome to Home Page',
  key: Key('homeGreetingLbl'),
)
Enter fullscreen mode Exit fullscreen mode
  • Add "enableFlutterDriverExtension()" before "runApp()"
void main() {
  enableFlutterDriverExtension();
  runApp(MyApp());
}
Enter fullscreen mode Exit fullscreen mode
  • Add "test" and "flutter-driver" to pubspec.yml
dev_dependencies:
  test: any
  flutter_test:
    sdk: flutter
  flutter_driver:
    sdk: flutter
Enter fullscreen mode Exit fullscreen mode

My tutorial was extended from TruongSinh Tran-Nguyen's tutorial, Big thank you to him. Installation can follow with his tutorial as below.

GitHub logo truongsinh / appium-flutter-driver

Appium Flutter Driver is a test automation tool for Flutter apps on multiple platforms/OSes. Appium Flutter Driver is part of the Appium mobile test automation tool.

Appium Flutter Driver

Build Status Greenkeeper badge NPM version Downloads Dependency Status devDependency Status

Appium Flutter Driver is a test automation tool for Flutter apps on multiple platforms/OSes. Appium Flutter Driver is part of the Appium mobile test automation tool.

⚠️ pre-0.1.x version

This package is in early stage of exprienment, breaking changes and breaking codes are to be expected! All contributions, including non-code, are welcome! See TODO list below.

Flutter Driver vs Appium Flutter Driver

Even though Flutter comes with superb integration test support, Flutter Driver, it does not fit some specific use cases, such as

  • writing test in other languages than Dart
  • running integration test for Flutter app with embedded webview or native view, or existing native app with embedded Flutter view
  • running test on multiple devices simultanously
  • running integration test on device farms, such as Sauce Labs, AWS, Firebase

Under the hood, Appium Flutter Driver use the Dart VM Service Protocol with extension ext.flutter.driver, similar to…

Let's test our app with appium-flutter-driver!

I've created demo application for our appium testing in github you can checkout and try

Appium-flutter-driver demo

Flutter example project for appium-flutter-driver.




Project has flutter app project and "appium" directory for testing. In "appium" directory consist of

  • package.json - Package which is required by appium-flutter-driver.
{
  "name": "appium",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "dependencies": {
    "appium-flutter-finder": "0.0.13",
    "webdriverio": "^5.15.0"
  },
  "scripts": {
    "start": "node test.js",
    "ios": "APPIUM_OS=ios npm start",
    "android": "APPIUM_OS=android npm start",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "TruongSinh Tran-Nguyen <i@truongsinh.pro",
  "private": true,
  "license": "MIT"
}
Enter fullscreen mode Exit fullscreen mode
  • test.js - This is main testing file.
const wdio = require('webdriverio');
const assert = require('assert');
const find = require('appium-flutter-finder');

const osSpecificOps = process.env.APPIUM_OS === 'android' ? {
  platformName: 'Android',
  deviceName: 'emulator-5554',
  // @todo support non-unix style path
  app: __dirname +  '/../build/app/outputs/apk/debug/app-debug.apk',
}: process.env.APPIUM_OS === 'ios' ? {
  platformName: 'iOS',
  platformVersion: '12.2',
  deviceName: 'iPhone X',
  noReset: true,
  app: __dirname +  '/../ios/Runner.zip',

} : {};

const opts = {
  port: 4723,
  capabilities: {
    ...osSpecificOps,
    automationName: 'Flutter'
  }
};

(async () => {
  console.log('Initial app testing')
  const driver = await wdio.remote(opts);
   assert.strictEqual(await driver.execute('flutter:checkHealth'), 'ok');
  await driver.execute('flutter:clearTimeline');
  await driver.execute('flutter:forceGC');

  //Enter login page
  await driver.execute('flutter:waitFor', find.byValueKey('loginBtn'));
  await driver.elementSendKeys(find.byValueKey('emailTxt'), 'test@gmail.com')
  await driver.elementSendKeys(find.byValueKey('passwordTxt'), '123456')
  await driver.elementClick(find.byValueKey('loginBtn'));

  //Enter home page
  await driver.execute('flutter:waitFor', find.byValueKey('homeGreetingLbl'));
  assert.strictEqual(await driver.getElementText(find.byValueKey('homeGreetingLbl')), 'Welcome to Home Page');

  //Enter Page1
  await driver.elementClick(find.byValueKey('page1Btn'));
  await driver.execute('flutter:waitFor', find.byValueKey('page1GreetingLbl'));
  assert.strictEqual(await driver.getElementText(find.byValueKey('page1GreetingLbl')), 'Page1');
  await driver.elementClick(find.byValueKey('page1BackBtn'));

  //Enter Page2
  await driver.elementClick(find.byValueKey('page2Btn'));
  await driver.execute('flutter:waitFor', find.byValueKey('page2GreetingLbl'));
  assert.strictEqual(await driver.getElementText(find.byValueKey('page2GreetingLbl')), 'Page2');
  await driver.switchContext('NATIVE_APP');
  await driver.back();
  await driver.switchContext('FLUTTER');

  //Logout application
  await driver.elementClick(find.byValueKey('logoutBtn'));
  driver.deleteSession();
})();
Enter fullscreen mode Exit fullscreen mode

Run Appium (Android)

  1. Change directory to "appium" folder.
  2. Start appium server with command "appium" in console.
  3. Start Android emulator (check your emulator address with "adb devices
Taweechais-MBP:~ taweechaimaklay$ adb devices
List of devices attached
emulator-5554   device
Enter fullscreen mode Exit fullscreen mode

install required nodejs package

npm install
Enter fullscreen mode Exit fullscreen mode

and change Android configuration in test.js") and run command

APPIUM_OS=android npm start
Enter fullscreen mode Exit fullscreen mode

to start appium automation.

Testing result

Automated running application with appium-flutter-driver
Appium run flutter app

Console output as below

Others Appium's command which I've tested.

//Enter value to tartget TextInput
await driver.elementSendKeys(find.byValueKey('telphoneEditTxt'), '1234567890')

//Wait for element visible
await driver.execute('flutter:waitFor', find.byValueKey('page2Btn'));

//Screen scrolling
await driver.execute('flutter:scrollIntoView', find.byValueKey('bottomSubmitBtn'), {alignment: 0.1});

//Using Native command and switch context back to flutter
await driver.switchContext('NATIVE_APP');
  driver.back();
await driver.switchContext('FLUTTER');

//Tap on widget element
await driver.elementClick(find.byValueKey('backBtn'));

Enter fullscreen mode Exit fullscreen mode

Conclusion

I did not have strong opinion about appium-flutter-driver is the best way to do flutter app automate testing. But it help QA or automation tester to get familiar with flutter app testing since they can reuse their Appium which already mature to start automate flutter app testing and they can lower their guard about flutter which is very new platform and not much test tooling for it. By the way for flutter developer I think flutter-driver is enough for automate testing (since they already know dart :)). That all from me, comment, question and suggestion is welcome.

Top comments (20)

Collapse
 
veenaharsha profile image
Veena Yemmiganur

Hello Taweechai,
How are you.
Thanks for the blog.
Am trying to set up the test automation for flutter application using Appium,
am new to Appium & flutter, am looking at setting up the testing project , preferably in javascript.
Thanks in advance

Collapse
 
atul161 profile image
atul anand

Is there any ways to automate google Authorization using appium ?
How can i test webview using this. library .
Thanks in Advance\

Collapse
 
joyce0101 profile image
joyce0101

I also encountered this problem. I want to test a hybrid application of flutter and webview. Have you found a good solution?

Collapse
 
zaphodikus profile image
Conrad Braam🤖

Can you explain this statement, but in a way a test engineer who does NOT write DART/Flutter code can go check and understand this? My team uses a framework that generates a lot of code, so the explanation assumes a but much of me. Giving controls a key or ID changes the kind of control or is the key "metadata" that the special driver hooks into?
"Since Flutter did not use native conponents, it's mean you need to define keys in all of your widget to made your widgets visible to appium-flutter-driver (also for normal flutter testing)."

Collapse
 
nitinvermaa profile image
Nitinvermaa

Hi Taweechai,
Hope you are doing good.

I'm getting the below issue while running nodejs project.
Node Version: 12.13.1
appium: 1.18.1

2020-09-02T19:43:13.369Z ERROR webdriver: Request failed due to Error: read ECONNRESET
at TCP.onStreamRead (internal/stream_base_commons.js:201:27)
(node:29568) UnhandledPromiseRejectionWarning: Error: read ECONNRESET
at TCP.onStreamRead (internal/stream_base_commons.js:201:27)
(node:29568) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by
rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:29568) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js pro
cess with a non-zero exit code.

Collapse
 
veenaharsha profile image
Veena Yemmiganur

Hi ,
Am also facing the same issue..
Did u get the solution for this issue

Collapse
 
fredrickpard profile image
Kodomo

Hey Taweechai, thanks for your tutorial. But, it still a bit difficult for me as a new user to understand the procedure for it.
Can you please create a demo video?
Thank you

Collapse
 
iabhay85 profile image
iabhay85

(node:74846) UnhandledPromiseRejectionWarning: Error: An unknown server-side error occurred while processing the command. Original error: Cannot execute command waitFor, server reponse {
"isError": true,
"response": "Uncaught extension error while executing waitFor: 'package:flutter_driver/src/extension/extension.dart': Failed assertion: line 373 pos 14: 'WidgetsBinding.instance!.isRootWidgetAttached || !command.requiresRootWidgetAttached': No root widget is attached; have you remembered to call runApp()?\n#0 _AssertionError._doThrowNew (dart:core-patch/errors_patch.dart:46:39)\n#1 _AssertionError._throwNew (dart:core-patch/errors_patch.dart:36:5)\n#2 FlutterDriverExtension.call (package:flutter_driver/src/extension/extension.dart:373:14)\n#3 BindingBase.registerServiceExtension. (package:flutter/src/foundation/binding.dart:597:32)\n\n",
"type": "_extensionType",
"method": "ext.flutter.driver"
}

Collapse
 
iabhay85 profile image
iabhay85

Getting below error can some one help here

Collapse
 
giangthao profile image
giangthao

I can't catch id, accessibility id, xpath after switch to NATIVE even though runInbackground worked my code await driver.switchContext('NATIVE_APP');
await driver.saveScreenshot('./native-screenshot.png');
driver.background(10)
// await driver.back();
await driver.pause(3000)
await (await await driver.$('//android.widget.ImageView[@content-desc="Project\n Tab 2 of 4"]')).click();

Collapse
 
hattersharath profile image
hattersharath

Hi Taweechai,

It was a great explanation. As I am a beginner, I wanted to check can we implement this using java?

Thank you
Sharath

Collapse
 
tempest1000 profile image
Tempest1000

Take a look at the following:

github.com/truongsinh/appium-flutt...

And this article which describes how to use Appium Flutter Finder and Java (use Google Translate browser plugin to convert the language):
cnblogs.com/hbolin/p/12510967.html

Collapse
 
netfirms profile image
Taweechai Maklay

I've found JavaScript only for now.

Collapse
 
tevaautomation profile image
Automation

Hey Taweechai, thanks for your tutorial. i couldn't understand if Appium fluuter driver works on real IOS device , not just on emulator or simulator.
is it possible?

Collapse
 
dpk7974 profile image
dpk7974

Hey Taweechai, still I am confused how to set up the flutter driver over eclipse, can you please create a demo video of Flutter app automation?

Collapse
 
netfirms profile image
Taweechai Maklay

I think you don't need any IDE to do testing setup.

Some comments may only be visible to logged-in visitors. Sign in to view all comments.