DEV Community

Cover image for Can Testers Use an Emulator for Closed Testing?
vmzavas
vmzavas

Posted on Originally published at peerplay.vmcreate.rs

Can Testers Use an Emulator for Closed Testing?

Sooner or later every developer stuck at 9 or 10 testers has the same idea. Android Studio can run as many virtual phones as your laptop can handle, and each one can sign in to a Google account. Why not fill the last few slots that way?

It's a fair question, and the honest answer has two parts: what the emulator can technically do, and what it does to your chances when you apply for production.

What an emulator can actually do
Android Studio's emulator comes with different system images. The ones marked Google Play include the Play Store, so you can sign in with a Google account, open the opt-in link, accept the invitation and install the closed test build just like on a phone.

Images without Google Play, such as the plain Google APIs or AOSP ones, can't do that. You can sideload an APK onto them for your own debugging, but that install doesn't go through Play and has nothing to do with your closed test.

So a tester who has no Android phone but uses Android Studio on a computer can, in principle, take part. That's the narrow case where an emulator is a reasonable answer.

Keep in mind what the emulator can't show you. It runs on your computer's processor and memory, so performance, battery use, camera behaviour and some sensors look different from a real phone. Bugs that only appear on a specific manufacturer's Android version won't show up either.

Why it's a risky way to reach 12
The rule for new personal developer accounts asks for at least 12 testers opted in to a closed test for 14 days in a row. The point behind it is that real people use your app before the public does. Google doesn't publish how it checks testers, so I can't tell you exactly what it sees about emulator installs, and I wouldn't bet a developer account on it seeing nothing.

The bigger problem is what you'd be doing. One person running several emulators with several Google accounts isn't testing with real users; it's manufacturing them. Google Play's policies are strict about fake engagement and about the accounts behind it, and the downside of getting that wrong is much worse than two more weeks of recruiting.

Even if nothing is flagged, you still have to apply for production. Play Console asks how you recruited testers, how they used the app and what you changed based on their feedback. Emulators don't give feedback, so you end up with weaker answers at exactly the moment they matter.

Where emulators do help
Emulators are great for your own testing. Run the release build on a few screen sizes and Android versions before you share the opt-in link, check that sign-up works, and make sure the app doesn't crash on a small, low-memory device profile. The fewer crashes your real testers hit on day one, the more of them stay for 14 days.

Play Console's pre-launch report is also worth a look. When you upload a build, Google can run it automatically on a set of test devices and report crashes and accessibility issues. It doesn't count toward your testers, but it's free feedback you didn't have to recruit for.

If a real tester only has a computer
Sometimes a genuine tester, often another developer, wants to help but uses an iPhone day to day. If they already run Android Studio, an emulator with Google Play is a reasonable way for them to take part. Ask them to sign in with the same Google account they gave you for the email list or Google Group, then opt in and install from the Play Store inside the emulator.

Be realistic about it, though. An emulator only runs while their computer is on and Android Studio is open, so they're much less likely to open your app regularly over 14 days than someone with it on their phone. Treat these testers as extras, not as part of your core 12.

If you rely on a few of them, check in during the test. A short message around day 5 and day 10 asking whether they've opened the app recently is enough to catch anyone who has drifted away.

Getting real testers instead
If you're short by a few people, ask your existing testers to bring one Android-using friend each, post in developer communities that do testing swaps, and recruit more than 12 so a dropout doesn't reset your progress.

I built PeerPlay after running into the 12-tester requirement with my own Android app. Developers test each other's apps in tracked 14-day rounds, and PeerPlay checks that a tester actually opened your app for a real session rather than just installing it. Testers who go quiet are removed from the round, so you see a drop early and can replace them with someone real.

Top comments (0)