<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Pierre Raffalli</title>
    <description>The latest articles on DEV Community by Pierre Raffalli (@pierridotite).</description>
    <link>https://dev.to/pierridotite</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4251657%2F7b0d7495-49f8-46a7-af50-65aa1e93789b.jpg</url>
      <title>DEV Community: Pierre Raffalli</title>
      <link>https://dev.to/pierridotite</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/pierridotite"/>
    <language>en</language>
    <item>
      <title>AI agents that beta-test mobile apps on 20 phones at once (and what they found in Flutter's sample)</title>
      <dc:creator>Pierre Raffalli</dc:creator>
      <pubDate>Wed, 18 Nov 2026 10:51:09 +0000</pubDate>
      <link>https://dev.to/pierridotite/ai-agents-that-beta-test-mobile-apps-on-20-phones-at-once-and-what-they-found-in-flutters-sample-2nf7</link>
      <guid>https://dev.to/pierridotite/ai-agents-that-beta-test-mobile-apps-on-20-phones-at-once-and-what-they-found-in-flutters-sample-2nf7</guid>
      <description>&lt;p&gt;I build a mobile app on my own. Unit tests pass, the app works on my phone, and then someone on an iPhone SE with large text finds a button hidden under the keyboard. I wanted something between unit tests and real beta testers, so I built this:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;One GitHub Actions workflow that builds your app, starts up to 20 iOS Simulators and Android emulators, and gives each one to an AI agent with a persona.&lt;/strong&gt; Each agent uses the app like a new user and writes a bug report with numbered screenshots.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fpierridotite%2Fmobile-agent-testers%2Fmain%2Fassets%2Fdemo.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fraw.githubusercontent.com%2Fpierridotite%2Fmobile-agent-testers%2Fmain%2Fassets%2Fdemo.gif" alt="Four AI testers using the same app at the same time: a small Android phone, a Pixel 7 in dark mode, an iPhone SE in French and an iPad with the largest text" width="560" height="315"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;It's open source (MIT): &lt;a href="https://github.com/pierridotite/mobile-agent-testers" rel="noopener noreferrer"&gt;github.com/pierridotite/mobile-agent-testers&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Update: 20 phones at once on LocalSend
&lt;/h2&gt;

&lt;p&gt;I then ran all 20 personas at once on &lt;a href="https://github.com/localsend/localsend" rel="noopener noreferrer"&gt;LocalSend&lt;/a&gt;, a 100k-star Flutter app: 10 Android emulators and 10 iOS Simulators, 14 languages, phones and tablets. 103 issues in 69 minutes, for $4.62 of model usage.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fthpljnwrs1p4zhxcca68.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fthpljnwrs1p4zhxcca68.png" alt="Four LocalSend screens with Flutter's overflow stripes, each boxed in red: a Pixel 7 at 200% text, an iPhone SE in Swedish at the largest text size, an iPad Air in German and an iPhone SE in French" width="800" height="475"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The worst one: on an iPhone SE at iOS's largest text size, the "type a message" dialog overflows by 149 px, so the text field and Confirm are off screen and you can't send text at all. At Android's 200% font, all six Send buttons overflow. The video of the 20 screens is in the &lt;a href="https://github.com/pierridotite/mobile-agent-testers" rel="noopener noreferrer"&gt;README&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  A real bug, found twice
&lt;/h2&gt;

&lt;p&gt;To check it on an app I didn't write, I pointed two agents at the example app of Flutter's official &lt;code&gt;animations&lt;/code&gt; package. One ran on a Pixel 7 with Android 14, the other on an iPhone SE in French with large text. Neither knew about the other.&lt;/p&gt;

&lt;p&gt;Both reported the same typo: the screen title says "Streamling your courses" instead of "Streamlining". It's still there on &lt;a href="https://github.com/flutter/packages/blob/main/packages/animations/example/lib/shared_axis_transition.dart#L110" rel="noopener noreferrer"&gt;line 110&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgb1psaftpvvjyrhro4dp.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgb1psaftpvvjyrhro4dp.png" alt="Both phones on the same screen of the Flutter sample, with the typo circled" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;A typo is small, but nobody writes a test for one. The same run also noted things a scripted test would not look for: the app ignores the system dark mode (the Android phone was in dark mode), and some light grey text is hard to read on a small screen.&lt;/p&gt;

&lt;p&gt;It also got one wrong: the iPhone agent flagged a narrow settings sheet as cut off when it was not. That is why every issue points to numbered screenshots, so you can check it in a few seconds.&lt;/p&gt;

&lt;p&gt;The run took 19 minutes, builds included, and $0.47 of model usage for both agents.&lt;/p&gt;

&lt;h2&gt;
  
  
  How it works
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Personas set the device.&lt;/strong&gt; Each persona is a few lines of JSON: phone model, OS version, font size, dark mode, language, location, and who the user is ("retired teacher, 68, low patience with jargon"). The repo ships four.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Each agent has three tools:&lt;/strong&gt; see the screen (a screenshot plus the accessibility tree with tap coordinates), gestures, and typing. Agents read pixels and the accessibility tree, so any UI toolkit works.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The build is automatic.&lt;/strong&gt; Flutter, React Native, Expo, native Kotlin and Swift, Kotlin Multiplatform, Capacitor and .NET MAUI are detected. For anything else, give your own build command or a prebuilt app.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Results are encrypted.&lt;/strong&gt; Logs of a public repo are public, so reports, screenshots and optional screen recordings are encrypted with your key, then decrypted on your machine.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Analytics are blocked&lt;/strong&gt; inside the phones, so test sessions never show up in your real stats.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What it costs
&lt;/h2&gt;

&lt;p&gt;GitHub Actions runners are free for public repos: about 20 jobs at once, 5 of them on macOS for the iPhones. The only real cost is the model. A Claude subscription works if you run batches of 6 to 10 agents, and an API key works too.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it is not
&lt;/h2&gt;

&lt;p&gt;It does not replace Maestro, Detox or Appium. Those replay the steps you wrote, the same way every time, which is what you want for regressions. These agents explore, so two runs never find exactly the same things. I use scripted flows for regressions and the agents before a release.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;gh repo create my-testers --public --template pierridotite/mobile-agent-testers --clone
&lt;span class="nb"&gt;cd&lt;/span&gt; my-testers &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; ./setup.sh        &lt;span class="c"&gt;# key pair, secrets, Claude token&lt;/span&gt;
gh workflow run agent-testers.yml  &lt;span class="c"&gt;# demo app: 2 Android + 2 iOS testers, about 20 minutes&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then point it at your own app with &lt;code&gt;-f source_repo=you/app -f app_id=com.you.app&lt;/code&gt;. Or install the Claude Code plugin and let Claude set it up:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;/plugin marketplace add pierridotite/mobile-agent-testers
/plugin install mobile-agent-testers@mobile-agent-testers
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If you run it on your app, I'd like to know what it finds, good or bad. Issues and feedback on the persona format are welcome.&lt;/p&gt;

</description>
      <category>testing</category>
      <category>ai</category>
      <category>flutter</category>
      <category>reactnative</category>
    </item>
  </channel>
</rss>
