Mobile App Automation with AWS Device Farm, Gitlab and Appium

Summary:
This post talks about how you can deploy and run your Appium Automation code into AWS Device Farm from Gitlab and your Release Platform eg. Visual Studio App Center.

WorkFlow:

1. Developer merges code to Gitlab Repo.
2. Repo will trigger Webhook to Release Platform to download code and build app for multiple platforms. (Android / iOS)
3. Once App file (.apk or .ipa) file is built Release Platform will send a Webhook notification to Gitlab to trigger Automation pipeline.
4. This pipeline will have 3 jobs:
   • Download and upload the installable build files (.apk or .ipa) to AWS Device farm.
   • Upload Latest Test Scripts to AWS Device Farm.
   • Execute Automation Run in Android and iOS, download test reports.

Stages

1. Setup Webhook in Release Platform to trigger Pipeline on Successful App Build.
2. Setup Job in Pipeline to download generated build and upload to AWS Device Farm.
3. Setup Job in pipeline to upload latest version of Appium test scripts to AWS Device Farm.
4. Trigger Automation Run in AWS Device Farm.

Setup Webhook in Release Platform.

• Create a trigger token in from your Gitlab account
  https://docs.gitlab.com/ee/ci/triggers/

• Construct the Webhook URL
  https://gitlab.example.com/api/v4/projects/<project_id>/ref/<ref_name>/trigger/pipeline?token=<token>

• In App Center if you go to Settings -> Webhooks, you can create your webhook with the above generated Gitlab pipeline trigger URL.
  https://appcenter.ms/orgs/<your organization>/apps/<your app>/settings/webhooks/create

• Configure when the webhook should be triggered.

• Configure webhooks for Android and iOS apps separately so that it does not conflict with each other.

Upload latest build to AWS Device Farm

• Create a Automation Test Project from AWS console in the Device Farm Service. Note down the Project ARN, Test Spec ARN etc.

• While setting up make sure the Test spec is as per your requirements. Make edits if needed.

• Once the Pipeline is trigger is configured, create a Gitlab Job, that executes a bash script.

• In the bash script, use App Center API via curl to get the the buildID for the latest build generated.
  $(curl -X GET "https://api.appcenter.ms/v0.1/apps/<your-organization>/<APP_NAME>/branches/develop/builds" -H "accept: application/json" -H "X-API-Token: <APIToken>")
  Here APIToken is App Center API Token. You can add this to Gitlab Environment Variables and consume it in the pipeline.

• Once the buildID is fetched you can download the build using
  $(curl -X GET "https://api.appcenter.ms/v0.1/apps/<your-organization>/<APP_NAME>/builds/<build_id>/downloads/build" -H "accept: application/json" -H "X-API-Token: <APIToken>")

• After downloading, you can use the aws cli command to upload it to the AWS Device farm.
  aws devicefarm create-upload --project-arn <device-farm-arn> --name <build-file-name> --type <extension>

Upload Test Scripts to Appium.

• Create a job in Gitlab pipeline that executes a bash script.

• In the bash script, navigate to the test folder and using npm-bundle plugin generate a zip file off the tests folder.
  

npm install -g npm-bundle
npm i
npm-bundle
zip -r appium-test.zip *.tgz

• Once the zip file is generated use aws cli command to upload it to the device farm.

aws devicefarm create-upload --project-arn <device-farm-arn> --name "appium-test.zip" --type APPIUM_NODE_TEST_PACKAGE

Trigger Automation Run in AWS Device Farm

• Once both the latest build (.apk or .ipa) file is successfully upload to the device farm. Create a python script with reference to this document.

https://docs.aws.amazon.com/devicefarm/latest/developerguide/api-ref.html

• You might have to pass the Device Farm Project ARN, Test Spec ARN, Device Pool ARN , App Name and Test Package Name as arguments to the python script. These ARN values are obtained from Device Farm Dashboard in AWS Console.
  Under Device Farm -> Mobile Device: Projects ->

• When this script is executed, it will trigger the automation run and once done, the report will be saved in Gitlab artifacts.

Here is an example screenshot:

Hope this document helps it setting up Mobile Automation in AWS Device Farm from Gitlab. Let me know if you have any questions in the comments.