Android runs on thousands of device models. As of 2026, there are over 24,000 distinct Android device models in active use. No QA team can test on all of them.
The question is not how to test on every device — it is how to make strategic choices about which devices to test on, so that the combinations you choose catch the bugs that actually affect your users.
That requires a test matrix built on data, not gut feeling.
Why Fragmentation Causes Bugs
Device fragmentation causes bugs through three mechanisms.
Hardware differences: Different processors (Snapdragon, Exynos, MediaTek), different GPUs, and different RAM configurations produce different performance profiles. Memory leaks that are invisible on a 12GB flagship cause crashes on a 2GB budget device. GPU rendering differences cause visual glitches on specific models. These bugs only appear on the affected hardware.
OEM customisations: Manufacturers modify Android. Samsung's One UI manages background tasks more aggressively than stock Android. Xiaomi's MIUI has different permission handling. Huawei's EMUI has a different power management model. An app that passes all tests on a Pixel can fail on a Samsung because of these differences — even on the same Android version.
OS version behavior: Each Android version introduces API changes, permission model updates, and behavioral differences. Android 11 introduced scoped storage. Android 13 added notification runtime permissions. Android 14 tightened background task restrictions. Apps must handle the version their users are actually running, not just the latest.
Step 1: Start With Your User Data
Before deciding which devices to test on, look at what your users actually have.
Open Google Play Console → Statistics → Android version distribution and Device model distribution. This shows exactly which device/OS combinations your users are running.
Find the combinations that represent 70% of your active users. This is your Tier 1 test matrix — the configurations that must pass before any release. If Samsung Galaxy A-series on Android 13 represents 25% of your users, that combination is in Tier 1 regardless of how inconvenient it is to test.
Step 2: Define Your Tiers
A practical device matrix has three tiers:
Tier 1 — Must test on physical devices or direct simulation
Your top 3-4 device/OS combinations by user share. These run your full test suite before every release. No exceptions.
Tier 2 — Should test before major releases
Device/OS combinations that represent another 20-30% of your user base. Run your critical path test suite (authentication, core workflows, payments) on these before major releases and version updates.
Tier 3 — Simulate or use cloud labs
Everything else. Use Firebase Test Lab, BrowserStack, or device simulation to cover this tier. You cannot own every device, but you can extend coverage through tools that give you access to configurations you don't have.
Step 3: Identify the High-Risk Combinations
Not all device/OS combinations are equally risky. Some configurations are more likely to produce bugs:
Minimum supported API level: Always test on the lowest Android version you support. This is where API compatibility bugs appear. If you support Android 10, test on Android 10.
Newest Android version: Test on the latest Android version as soon as you update your target SDK. New behavioral changes appear here.
Budget devices: Low RAM devices expose memory issues invisible on flagships. A Motorola G-series or Samsung A-series on 2-3GB RAM should be in your Tier 1 if budget device users are part of your audience.
OEM heavy markets: If your analytics show significant Samsung or Xiaomi usage, those OEM skins need dedicated testing. Stock Android behavior does not predict behavior on One UI or MIUI.
Step 4: Reproducing Device-Specific Bugs Without Owning the Device
When a bug is reported on a specific device model you don't own, you need a way to reproduce it. This is where device simulation becomes practical.
The key is matching the device profile accurately: device identifiers, build fingerprint, system properties, and Android version. When a tester reports a crash on a Samsung Galaxy A34 running Android 14, recreating those environment conditions allows developers to reproduce the bug without owning that device.
Document the profile used to reproduce every device-specific bug. This creates a library of reproduction profiles that the whole team can use, making bug verification and regression testing significantly faster.
Keeping the Matrix Current
Your user base changes over time. Android version distribution shifts as users update their devices. New device models gain market share.
Review your test matrix every quarter by pulling fresh Play Console analytics. Add new combinations that have grown to significant share. You can generally drop combinations that have fallen below 1% of your active users.
The full device fragmentation testing guide — including the matrix-building framework, coverage gap analysis, and how to use simulation to extend beyond your physical device lab — is at deviceschanger.org/android-device-fragmentation-testing.
Top comments (0)