On average, people spend 6 hours and 37 minutes a day looking at their phone. Today, the internet traffic from mobile has surpassed desktops signifying how people consider mobile devices as not just devices to connect to the world but an integral part of their lives, especially for millennials who spend 25% of their waking lives using a mobile phone.
Mobile manufacturers have not sidelined this fact and have understood that releasing newer mobile devices is much more profitable than releasing newer laptops at the same pace.
However, with each new device release, mobile app testers require one more testing device and more test cases associated with it. This creates an issue of procurement, setting up, and maintenance of thousands of devices that keep increasing each day. It is certainly a problem in the mobile world and the best solution to it is – using device farms!
What are device farms?
The solution to the deeply fragmented mobile world we live in is device farms. As vegetable farms have vegetables, and poultry farms have poultry, a device farm contains devices already procured and placed at a certain physical location. These are real devices that are unique in one way or another. For instance, one may find two Samsung Galaxy S24 in a device farm but both of them may have different configurations (like RAM or ROM). These devices can be accessed either physically by going to a location or through the internet. However, most of the device farms allow only virtual access.
The main aim of a device farm is to satiate the requirements of mobile app testing which majorly consists of Android testing and iOS testing. It eliminates the need to purchase these devices and allows the testers to select a device, upload their application, and just start to operate on it instantly without spending too much money from their pocket. They certainly solve the biggest challenge of mobile app testing and, today, there are hardly any organizations that do not make use of it.
The need and market for device farms exploded with the outburst of mobile device sales and the number of applications being developed for different operating systems. Such high demands led to the inception of device farms by various companies providing a twist of features to lure the testers. However, to select the one best for us, we need to explore these device farms one by one.
Top device farms for Android and iOS testing
The following device farms serve as the best options in the current times with lucrative features and impressive performance.
TestGrid
One of the top mobile device farms that not only lets you connect in a few seconds but also provides the hardware settings control to the testers is TestGrid. The platform spans over vast variety of testing domains providing a single-stop solution for all types of testers and their testing requirements. It provides real devices with their farm constantly upgrading with newer ones from time to time.
The most attractive features of TestGrid are as follows:
Device lock-in facility: Most device farms will reassign the device once the tester’s session closes or they log out of their account. However, TestGrid provides a facility to lock in a device for a certain period allotting the device completely to the tester.
Real device support: TestGrid provides real devices for testing that are placed on their premise ready to begin the testing work.
Beyond mobile application: Most device farms will open just the application after installation and will allow testing only within that boundary. With TestGrid, the tester gets complete control of the device with which they can also make hardware settings change and navigate across the device as they would on a real device.
Wide variety of testing: TestGrid supports a wide variety of testing domains including performance testing, API testing, cross-browser testing, automation testing, and visual testing.
Codeless support: Organizations adopting a codeless testing paradigm, can achieve the same using TestGrid’s record-and-play feature without writing any line of code.
Biometric support: TestGrid device farm supports biometric functionality testing that includes facial recognition and fingerprint verification through their real device.
Deep integration: The testers get integrated software such as JIRA and those that facilitate the CI/CD pipeline such as Jenkins.
Bottom line: TestGrid is a fast, robust, and economical device farm for Android as well as iOS. It provides a wide spectrum to work on with codeless support for all types of testers. It can be a primary choice for both individual testers and enterprises with large testing teams.
Pricing:
Freemium: Free forever (200 minutes/2 minute sessions)
Manual/Scriptless Automation: $25/mo (5 users, 1 parallel test)
End-to-End Automation: $99/mo (5 users, 1 parallel test)
Private Dedicated: $30/mo (5 users, 1 dedicated device)
Enterprise: Contact sales
AWS Device Farm
Established in 2015, the AWS device farm currently holds more than 2500 devices, desktop, Android, and iOS, with 12 months of free usage. It allows the testers to accomplish automation testing with real devices either by using built-in frameworks that do not require writing or maintaining tests or through supported frameworks directly.
Once the tests are completed, the testers get the results in the form of videos, logs, images, and other analytics for careful consideration. AWS device farm also allows geolocation settings and a smooth integration facility into the current development workflow for a seamless experience.
Bottom line: AWS is a great device farm with a feasible option to get started. Since it is hosted by AWS which also has its server farm and network hosting abilities, the testers can get a good benefit out of it if they are already using AWS components in other pieces of development.
Pricing – Custom based on test runs and infrastructure
Samsung Remote Test Lab
To satiate the physical device requirements and run the Android application, Samsung has its own Android device farm called Samsung Remote Test Lab. This lab consists of real Samsung devices including all the latest ones located at 10 locations in 8 countries. Once the tester signs up for free, the nearest location is connected. However, the tester is not restricted to using just that location and can connect to other service locations as well.
Samsung Remote Test Lab has a unique way of installing the Android application using the drag-and-install feature where the tester simply needs to drag the file from local explorer to their dashboard to install. Once done, they get a list of features to leverage on these devices. A few of them are as follows:
Setting alterations: Samsung Remote Test Lab provides a lot of setting options through which the testers can change the settings and test with which they are most comfortable.
Automation testing support: The device farm by Samsung comes with support for automation testing with a recording of integral parameters such as CPU usage and memory usage.
Multi-touch support: The testers can test their application for multi-touch gestures including zooming on the screen.
Audio streaming: The device farm for Android also supports audio streaming so the tester can listen to each sound embedded in their application.
Bottom line: Samsung Remote Test Lab is fast, user-friendly, and efficient in testing providing a user experience similar to a real device. However, the tester can use only Samsung devices in the lab and since they are manufactured by Samsung, they support Android applications only.
Pricing – Free to use for Samsung developer members.
Firebase Test Lab
Google’s answer to the problem of real device procurement for testing is Firebase Test Lab. Established for Android and iOS application testing, Firebase Test Lab provisions a real device (or a virtual device as per requirement) with just a free sign-up.
Firebase Test Lab can integrate tests with Firebase console, Android Studio, and gcloud CLI due to the reason that it is made by Google. It also provides support for the Robo test through which testers can create tests without writing any code by using the Robo driver that crawls the applications and does all the necessary work. Although, that does not mean test scripting is not allowed. Testers are free to choose the path they want and even integrate the CI pipeline directly through the command line into the lab. Once done, detailed reports are generated containing analytics and screenshots.
Bottom line: Firebase Test Lab can get one onboard quickly as compared to many other alternatives. It also provides support for iOS testing but with limited options. However, the downside is that Firebase Test Lab is not suitable for all testing domains that a tester may require. It can be used with other labs or frameworks but not exclusively.
Pricing: Spark plan (10 test runs on virtual device and 5 test runs on physical device per day) and Blaze plan ($5 per hour post free 30 minutes on physical device and $1 per hour post 60 minutes on virtual device)
Kobiton
Another piece in top device farms for Android and iOS testing is Kobiton. Similar to TestGrid, Kobiton also provides codeless testing capabilities however its device cloud only supports mobile and that too those that are popular (with popular operating systems).
Kobiton comes with a feature of device lab management that essentially means that a tester can create a lab of his own and select the target device for their application in that lab. Since it is a personal device lab, they no longer would be required to keep track of these devices and the lab will serve as an isolated one yet connected to the Kobiton ecosystem. One may also find a few elements of artificial intelligence embedded in scripting to make test case writing faster. This includes self-healing as well as Appium test case generation.
Bottom line: Kobiton device farm is an AI-supported tool with codeless support. However, considering the shortfalls of real devices, it may not be completely suitable for the organizational level of testing.
Pricing (As per plan): $83/$399/$9000/Custom
App Center Test
Microsoft’s device farm for Android and iOS testing is called App Center Test (formerly Xamarin Test Cloud). It supports testing frameworks that can be used for writing scripts and running them over the cloud on real devices hosted in the lab.
App Center Test provides extensive reporting with logs and screenshots. The testers can also schedule parallel runs i.e. execute tests on multiple devices at the same time in just a few steps.
Bottom line: Microsoft’s device farm is considerable only if the application does not have complex hardware-related features and the testers work on the selected frameworks. The device farm does not support manual testing, network simulation, VPN, or load testing.
Pricing: 30 days free trial post that per concurrent device.
Perfecto
A device farm with biometric support with network virtualization is Perfecto. Supporting both real devices and virtual devices, Perfecto allows a blend of scriptless and scripted testing for an efficient cycle.
Perfecto is also capable of expanding its span beyond simple UI testing to performance testing, API testing, and user experience testing. With a certified cloud system and high-class monitoring abilities, Perfecto is easily one of the top device farms for Android and iOS testing.
Bottom line: Perfecto is a great tool that allows most types of testing. However, the device farm lacks integrations and does not support hardware configurations like testing the camera, etc.
Pricing: $83/$155/Custom
Sauce Labs
A device farm that can take care of all your needs from development to post-release is Sauce Labs. It provides great support for automation software and integrates third-party apps as well.
Sauce Labs has been innovative for years providing solutions that could facilitate the testing cycles. For instance, it provides parallel testing and can open the code inspector to debug the code just as on a browser. It also provides simulators and emulators based on requirements along with real devices that can monitor the test executions and generate an extensive report later on.
Bottom line: Sauce Labs is a great platform that provides all the required functionalities along with CI/CD integration to the customers. However, the only downside is difficulty in accessing the hardware parameters and it does tend to be on a pricier side as compared to its alternatives.
Pricing: $249 per month for real devices, or $199 per month for virtual devices.
Headspin
Promoting itself as a Global Device Infrastructure, Headspin is present at more than 90 locations whose greatest impact is seen in geolocation testing.
Headspin supports both manual and automation testing on real devices with a primary focus on simulating real-world conditions no matter what type of tests are run. Its support for multiple automation frameworks makes it one of the top device farms for Android and iOS testing. Headspin also supports creating your own lab to organize the target devices in a single place.
Bottom line: Headspin is a great choice for testers targeting geolocation testing or in any other way requiring their application to run from different locations. However, Headspin may seem to be weaker in other testing domains and does not support a few mandatory ones such as API testing. It has also been conceived as an expensive tool.
Pricing: Based on requirements only.
Conclusion
As time goes by, the frequency with which manufacturers release their phones also increases. Today, more than 81% of Americans own smartphones and this percentage keeps increasing in each country. However, this serves as a problem for the developers and testers. The more devices there are, the more flexible our code needs to be to cater to all those devices. Hence, we need a solution that can provide us with devices and still be able to eliminate the overheads that come with it. This is exactly what device farms do.
Device farms are a collection of devices (mostly real) that sit at a physical location and are connected to the internet for usage. They are primarily designed to cater to the development and testing requirements although, that is not a restriction and they can be used for personal use as well. Device farms often try to differentiate themselves by providing unique features to the testers. Some like TestGrid can span across all testing domains while some like App Center Test have restrictive capabilities. This makes this top device farms list for Android and iOS testing worth reading and understanding which one suits our needs the best. Hope it helps in future projects and let us know your favorite tool in the comments below
This blog is originally published at TestGrid
Top comments (0)