If you've been running OpenSTF (Open Smartphone Test Farm) for your mobile testing infrastructure, you've probably hit a wall. Your Android 14 and 15 devices aren't working. Updates stopped years ago. And you're stuck wondering: what now?
You're not alone. Thousands of QA teams and developers built their device labs on OpenSTF, and now they're scrambling for alternatives that actually work with modern devices.
What Happened to OpenSTF?
OpenSTF was originally developed at CyberAgent in Japan to manage their growing collection of 160+ test devices. It became the go-to open-source solution for teams wanting to build their own device labs.
But in July 2020, the core team announced they were abandoning the project:
"This project along with other ones in OpenSTF organisation is provided as is for community, without active development."
The last official OpenSTF release was v3.4.1, which only supports up to Android 9. If you're trying to test on Android 10, 11, 12, 13, 14, or 15 devices—you're out of luck.
The DeviceFarmer Fork: Not the Answer
Development supposedly moved to a community fork called DeviceFarmer. But here's the reality:
- 156+ open issues sitting on GitHub
- Volunteer-maintained with no funding or dedicated team
- Slow Android version support—Android 14/15 support is incomplete
- Still no iOS support—same limitation as original OpenSTF
- Same ancient tech stack—RethinkDB, Node.js 8.x, ZeroMQ
The project openly acknowledges: "Development is still largely funded by individual team members and their unpaid free time, leading to slow progress."
One developer on Hacker News captured the sentiment perfectly: "They dropped support on it and moved to another product (DeviceFarmer) which never seems to have materialised at all. We just kept running OpenSTF until it stopped working."
Why OpenSTF's Architecture Was Always Flawed
1. The 12-Container Problem
To run OpenSTF in production, you needed to orchestrate:
- RethinkDB (database)
- nginx (reverse proxy)
- stf-app (web interface)
- stf-auth (authentication)
- stf-processor (device processing)
- stf-triproxy-app (message routing)
- stf-triproxy-dev (device routing)
- stf-reaper (cleanup)
- stf-storage-plugin-apk
- stf-storage-plugin-image
- stf-storage-temp
- stf-websocket
- stf-api
- stf-provider (one per host machine)
- adbd (ADB daemon)
Setting this up took days. Maintaining it was a part-time job.
2. No Data Privacy Architecture
The OpenSTF team acknowledged this in their own docs:
"We have made certain assumptions about the trustworthiness of our users. As such, there is little to no security or encryption between the different processes."
And:
"Devices do not get completely reset between uses, potentially leaving accounts logged in or exposing other sensitive data."
3. iOS Support: Never Happening
iOS support exists through a separate project (stf_ios_support), but it's barely functional:
- Only 1 iOS device per Mac machine
- 1 FPS hardcoded framerate
- Clicking is slow due to Apple API limitations
- Devices frequently get stuck in "preparing" state
- WebDriverAgent has memory leaks requiring restarts every 4 hours
4. Test Automation Gap
Here's what surprises many teams: OpenSTF is not a test automation platform.
OpenSTF provides remote device access—you can view the screen, tap, type, and install apps. But it doesn't run your Appium tests, Espresso suites, XCUITest cases, or Maestro flows.
The Modern Alternative: P2P Architecture
After running enterprise device labs for 12+ years serving companies like Disney+Hotstar, Airtel, Swiggy, and Jio, we saw the gap clearly:
- Small teams have the same security concerns as enterprises
- Cloud device farms require trusting third parties with sensitive data
- OpenSTF is dead and DeviceFarmer isn't a real solution
Zero-Trust Architecture Difference
OpenSTF Architecture:
Your Device → OpenSTF Server → Your Browser
(all data flows through central server)
Cloud Device Farm Architecture:
Your Test Data → Cloud Provider Servers → Test Results
(they promise not to look)
P2P Architecture:
Your Device ↔ WebRTC P2P ↔ Your Browser
(data flows directly, never touches third-party servers)
With P2P, your test data, app binaries, and device interactions flow directly between your devices and your browser via WebRTC peer-to-peer connections.
This isn't "we promise not to look at your data." This is "we architecturally cannot access your data."
What Modern Solutions Should Solve
| Problem | OpenSTF | Modern Solution |
|---|---|---|
| Android 14/15 support | Broken | Full support |
| iOS support | Never worked well | Native support |
| Data privacy | No encryption | Zero-trust P2P |
| Infrastructure complexity | 12+ containers | Single agent install |
| Maintenance burden | Significant | Managed updates |
| Test automation | Separate setup | Built-in |
Full Test Framework Support
Unlike OpenSTF which only provides remote access, modern solutions should support all major frameworks:
| Framework | OpenSTF | Modern Solution |
|---|---|---|
| Appium | Requires separate setup | Built-in |
| Espresso | Manual integration | Built-in |
| XCUITest | Complex setup | Built-in |
| Maestro | Not supported | Native support |
Migration from OpenSTF
If you're currently running OpenSTF or DeviceFarmer, migration should be straightforward:
What You Keep
- Your physical devices
- Your Appium test suites
- Your Espresso and XCUITest projects
- Your CI/CD integrations
- Your USB hubs and host machines
What You Gain
- iOS device support
- Android 14/15 support (and future versions)
- All test frameworks built-in
- Zero-trust data privacy
- No more RethinkDB maintenance
- No more minicap/minitouch debugging
What You Leave Behind
- RethinkDB administration
- Node.js 8.x security vulnerabilities
- The anxiety of running abandoned software
- Android-only limitations
- 12+ Docker containers
- Manual Appium server setup
When OpenSTF Still Makes Sense
To be fair, OpenSTF/DeviceFarmer isn't wrong for everyone:
OpenSTF might fit if you:
- Have dedicated DevOps resources with STF expertise
- Only need Android devices (older versions)
- Don't need integrated test automation
- Have unlimited time for setup and maintenance
- Run in an isolated network with no security requirements
Frequently Asked Questions
Is OpenSTF still maintained?
No. The original OpenSTF project was abandoned in July 2020. The last official release (v3.4.1) only supports Android 9. Development moved to DeviceFarmer, which is volunteer-maintained with limited resources.
What testing frameworks should modern solutions support?
Look for built-in support for Appium, Espresso, XCUITest, and Maestro. Your existing test suites should work without modification.
How should data privacy be handled?
Look for WebRTC peer-to-peer connections. Your device screen, inputs, and test data should flow directly between your device and your browser—never through third-party servers.
Can I migrate gradually from OpenSTF?
Yes. You can run new solutions alongside your existing OpenSTF setup during evaluation. Many teams migrate device-by-device over a few weeks.
The Bottom Line
OpenSTF was revolutionary for its time. It proved that teams could build their own device labs instead of paying cloud testing premiums. But it's 2025, and OpenSTF is dead.
The DeviceFarmer fork is on life support. Cloud alternatives require trusting third parties with your data.
Modern solutions should offer:
- WebRTC P2P architecture (not 2015-era message queues)
- Zero-trust privacy (your data never touches third-party servers)
- iOS and Android support
- All frameworks built-in (Appium, Espresso, XCUITest, Maestro)
- Actual maintenance and support
If you're still running OpenSTF or DeviceFarmer, it's time to upgrade.
We built DeviceLab to solve these exact problems. First device free forever.
Top comments (0)