DEV Community: Kutay VURAL

Testing SMS verification in mobile apps with Appium and Maestro

Kutay VURAL — Tue, 27 Oct 2026 12:00:00 +0000

SMS verification is everywhere in mobile apps, and it's also where E2E tests most often give up on it. This last post in the series covers Appium and Maestro. (Earlier parts covered Playwright, Cypress and Selenium; see the series list above.)

First, a misconception

The mobile app doesn't send the SMS. Your backend does: the app calls something like POST /auth/send-code, and the backend calls Twilio, Infobip, AWS or whichever provider you use.

That's good news for testing. You don't need to touch the device or the app binary at all. In the test or staging environment, point the backend's SMS SDK at a mock inbox (one option, only when OTPMOCK_URL is set). Nothing reaches a carrier, and the test reads the code over HTTP.

It also means SMS autofill (iOS one-time code suggestions, Android SMS Retriever) won't kick in during these tests, because no SMS reaches the device. Your test types the code, which is exactly what a user without autofill does. If you specifically want to test autofill, that needs a real SIM and a separate, small check.

Appium (WebdriverIO, TypeScript)

Use the small dependency-free helper (curl -O https://otpmock.com/sdk/otpmock.ts):

// test/specs/signup.e2e.ts
import { OtpMock } from "../support/otpmock";

const otp = new OtpMock({
  baseUrl: process.env.OTPMOCK_URL ?? "https://api.otpmock.com",
  apiKey: process.env.OTPMOCK_API_KEY!,
});

describe("phone verification", () => {
  it("signs up with an SMS code", async () => {
    const phone = otp.randomPhone(); // +1555XXXXXXX, unique per test

    await $("~phone-input").setValue(phone);
    const since = Date.now() - 5_000; // margin for clock skew
    await $("~send-code-button").click();

    const { code } = await otp.waitForCode(phone, { since, timeout: 20_000 });

    await $("~code-input").setValue(code);
    await $("~verify-button").click();
    await expect($("~welcome-screen")).toBeDisplayed();
  });
});

The ~ selectors are accessibility IDs, which work on both Android and iOS.

Appium (Python)

import os, random, time, urllib.parse
import requests
from appium.webdriver.common.appiumby import AppiumBy

API = os.environ.get("OTPMOCK_URL", "https://api.otpmock.com")
KEY = os.environ["OTPMOCK_API_KEY"]

def wait_for_code(phone, since_ms, timeout=20):
    url = f"{API}/v1/inbox/{urllib.parse.quote(phone)}/code"
    deadline = time.time() + timeout
    while time.time() < deadline:
        r = requests.get(url, params={"since": since_ms}, headers={"Authorization": f"Bearer {KEY}"})
        if r.status_code == 200:
            return r.json()["code"]
        if r.status_code != 404:
            r.raise_for_status()
        time.sleep(0.3)
    raise TimeoutError(f"no code for {phone}")

def test_phone_signup(driver):
    phone = "+1555" + "".join(random.choices("0123456789", k=7))
    driver.find_element(AppiumBy.ACCESSIBILITY_ID, "phone-input").send_keys(phone)
    since = int(time.time() * 1000) - 5000
    driver.find_element(AppiumBy.ACCESSIBILITY_ID, "send-code-button").click()

    code = wait_for_code(phone, since)
    driver.find_element(AppiumBy.ACCESSIBILITY_ID, "code-input").send_keys(code)
    driver.find_element(AppiumBy.ACCESSIBILITY_ID, "verify-button").click()

Maestro

Maestro flows are YAML, but runScript lets you run JavaScript with a built-in http client, which is all we need.

One script creates a unique number:

// scripts/new-phone.js
var digits = '';
for (var i = 0; i < 7; i++) digits += Math.floor(Math.random() * 10);
output.phone = '+1555' + digits;
output.code = '';

Another makes one attempt to read the code:

// scripts/fetch-code.js
var url = OTPMOCK_URL + '/v1/inbox/' + encodeURIComponent(PHONE) + '/code?since=' + SINCE;
var response = http.get(url, { headers: { 'Authorization': 'Bearer ' + OTPMOCK_API_KEY } });

if (response.status === 200) {
  output.code = json(response.body).code;
} else if (response.status === 404) {
  // Not there yet. Maestro scripts have no sleep, so wait briefly before the next attempt.
  var until = Date.now() + 500;
  while (Date.now() < until) {}
} else {
  throw new Error('otpmock ' + response.status + ': ' + response.body);
}

And the flow repeats that attempt until the code is there:

appId: ${APP_ID}
env:
  OTPMOCK_URL: https://api.otpmock.com
---
- launchApp:
    clearState: true
- runScript: ../scripts/new-phone.js
- tapOn:
    id: "phone-input"
- inputText: ${output.phone}
- evalScript: ${output.since = Date.now() - 5000}
- tapOn:
    id: "send-code-button"

- repeat:
    times: 30
    while:
      true: ${output.code == ''}
    commands:
      - runScript:
          file: ../scripts/fetch-code.js
          env:
            OTPMOCK_URL: ${OTPMOCK_URL}
            OTPMOCK_API_KEY: ${OTPMOCK_API_KEY}
            PHONE: ${output.phone}
            SINCE: ${output.since}
- assertTrue: ${output.code != ''}

- tapOn:
    id: "code-input"
- inputText: ${output.code}
- tapOn:
    id: "verify-button"
- assertVisible:
    id: "welcome-screen"

Run it with the key passed as an env var:

maestro test -e APP_ID=com.example.app -e OTPMOCK_API_KEY="$OTPMOCK_API_KEY" flows/signup-otp.yaml

React Native and Detox

Same story: the RN app calls your backend, the backend's SDK points to the mock in test environments, and the Detox (Jest) test uses the same helper as the Appium example. I wrote it up here. One exception: if you use Firebase Phone Auth, Firebase sends the SMS itself, so use Firebase's own test numbers or emulator instead.

Device farms and parallel runs

BrowserStack, Sauce Labs, AWS Device Farm: none of them need access to the mock. Only the test runner calls the inbox API, and each test has its own number, so you can run as many devices in parallel as you like.

Wrapping up the series

Across Playwright, Cypress, Selenium and mobile, the idea never changed: unique number, since, poll, type. The only thing that changes is how each tool makes an HTTP call.

I use otpmock for this, which I built after running into the problem one too many times. It emulates 15 SMS providers and their Verify APIs, with setups tested in seven languages. But whatever you use, the OTP step really doesn't have to be the untested part of your login flow.

Full guides: Appium, Maestro.

Testing SMS OTP flows in Selenium with Python and Java

Kutay VURAL — Tue, 20 Oct 2026 12:00:00 +0000

This series is about testing SMS OTP flows end to end without real text messages. After Playwright and Cypress (see the series list above), this one is for Selenium, which still runs a huge share of the world's E2E suites, often in Python or Java rather than JavaScript.

The good news: the pattern is the same, and the only tool-specific part is a tiny polling function.

The pattern in 30 seconds

In the test environment, the app's SMS provider SDK points to a mock inbox instead of the real provider (one option on the client, only when OTPMOCK_URL is set). Nothing reaches a carrier.
Each test uses its own fake number, so parallel sessions on a Grid never read each other's codes.
Take a since timestamp right before triggering the SMS.
Poll the inbox over HTTP for a code sent to that number after since, then type it in.

The inbox API is one request:

GET https://api.otpmock.com/v1/inbox/{phone}/code?since={unix_ms}
Authorization: Bearer $OTPMOCK_API_KEY

200 {"code":"482913","messageSid":"SM…","body":"Your code is 482913","receivedAt":1791417652342}
404 {"error":"no_code_yet"}

Python

# tests/otp.py
import os, random, time, urllib.parse
import requests

API = os.environ.get("OTPMOCK_URL", "https://api.otpmock.com")
KEY = os.environ["OTPMOCK_API_KEY"]

def random_phone():
    return "+1555" + "".join(random.choices("0123456789", k=7))

def wait_for_code(phone, since_ms, timeout=15):
    url = f"{API}/v1/inbox/{urllib.parse.quote(phone)}/code"
    deadline = time.time() + timeout
    while time.time() < deadline:
        r = requests.get(url, params={"since": since_ms}, headers={"Authorization": f"Bearer {KEY}"})
        if r.status_code == 200:
            return r.json()["code"]
        if r.status_code != 404:
            r.raise_for_status()
        time.sleep(0.3)
    raise TimeoutError(f"no code for {phone}")

# tests/test_signup.py
import time
from selenium.webdriver.common.by import By
from otp import random_phone, wait_for_code

def test_phone_signup(driver):
    phone = random_phone()
    driver.get("https://staging.example.com/signup")
    driver.find_element(By.NAME, "phone").send_keys(phone)
    since = int(time.time() * 1000) - 5000  # margin for clock skew
    driver.find_element(By.CSS_SELECTOR, "button[type=submit]").click()

    code = wait_for_code(phone, since)
    driver.find_element(By.NAME, "code").send_keys(code)
    driver.find_element(By.CSS_SELECTOR, "button[type=submit]").click()

Java

Plain java.net.http, no extra dependencies:

// OtpMock.java
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
import java.util.regex.*;

public final class OtpMock {
  private static final HttpClient HTTP = HttpClient.newHttpClient();
  private static final String API = System.getenv().getOrDefault("OTPMOCK_URL", "https://api.otpmock.com");
  private static final String KEY = System.getenv("OTPMOCK_API_KEY");

  public static String waitForCode(String phone, long sinceMs) throws Exception {
    String url = API + "/v1/inbox/" + URLEncoder.encode(phone, StandardCharsets.UTF_8) + "/code?since=" + sinceMs;
    long deadline = System.currentTimeMillis() + 15_000;
    while (System.currentTimeMillis() < deadline) {
      HttpResponse<String> r = HTTP.send(
          HttpRequest.newBuilder(URI.create(url)).header("Authorization", "Bearer " + KEY).build(),
          HttpResponse.BodyHandlers.ofString());
      if (r.statusCode() == 200) {
        Matcher m = Pattern.compile("\"code\":\"([^\"]+)\"").matcher(r.body());
        if (m.find()) return m.group(1);
      } else if (r.statusCode() != 404) {
        throw new IllegalStateException("otpmock " + r.statusCode() + ": " + r.body());
      }
      Thread.sleep(300);
    }
    throw new IllegalStateException("no code for " + phone);
  }
}

Swap the regex for Jackson or Gson if you already have one on the classpath.

Pointing the app at the mock (Python and Java backends too)

The app side isn't tied to JavaScript either. If your backend is in Python or Java, the provider SDK has a host option or accepts a custom HTTP client. A few examples:

# Python, AWS SNS with boto3
sns = boto3.client("sns", endpoint_url=os.environ.get("OTPMOCK_URL"))

// Java, Vonage
VonageClient.Builder builder = VonageClient.builder().apiKey(apiKey).apiSecret(apiSecret);
if (System.getenv("OTPMOCK_URL") != null) builder.httpConfig(HttpConfig.builder().baseUri(System.getenv("OTPMOCK_URL")).build());

I keep tested setups for 15 providers in Node, Python, Java, C#, PHP, Ruby and Go here.

Selenium Grid and parallel runs

Every test generates its own number, so you can scale Grid nodes without coordination. Only the test runner talks to the inbox API; the browser nodes never do, so they don't need network access to it.

Pitfalls

time.sleep(10) instead of polling. Codes usually arrive in well under a second. Polling is faster and less flaky.
Variables only on the runner. The app sends the SMS, so it needs OTPMOCK_URL too.
Number format. If the UI adds a country code or strips the +, read the code for the number the app actually sent to.

Wrapping up

I use otpmock for this, but any mock inbox you can query over HTTP works with the same code. Full guide: Selenium SMS OTP testing.

Next and last in the series: mobile apps with Appium and Maestro.

How I test SMS OTP flows in Cypress (the cy.task trick)

Kutay VURAL — Tue, 13 Oct 2026 12:00:00 +0000

In the previous post I showed how I test SMS OTP flows in Playwright without sending real texts. A few people asked about Cypress, and Cypress needs one extra trick, so here it is.

Quick recap of the idea: in the test environment, the app still sends SMS through its normal provider SDK, but the SDK points to a mock inbox instead of the real provider. Nothing reaches a carrier. The test reads the code over HTTP and types it in. No 000000 backdoor in the app, and the real flow gets tested.

Why Cypress is a bit different

In Playwright you can just await a fetch from the test. In Cypress, test code runs in the browser and is built around the command queue. Polling an external API from a spec is possible with cy.request and recursion, but it gets ugly fast, and CORS can get in the way.

The cleaner way: do the polling in Node, inside a cy.task. Tasks run in the Cypress Node process, so you can use a normal async helper with fetch, and the spec just waits for the result.

Step 1: point the app's SMS client at the mock

This part is the same as in any framework and lives in the app, not in Cypress. One option on the client you already create, only when OTPMOCK_URL is set. For example with the Vonage SDK:

import { Vonage } from "@vonage/server-sdk";

export const vonage = new Vonage(
  { apiKey: process.env.VONAGE_API_KEY, apiSecret: process.env.VONAGE_API_SECRET }, // secret = mock key in tests
  process.env.OTPMOCK_URL ? { restHost: process.env.OTPMOCK_URL, apiHost: process.env.OTPMOCK_URL } : {},
);

Twilio, AWS SNS, Infobip, Sinch and the others each have a similar one-liner (base URL, endpoint or host option).

Step 2: the task

Download the small helper (no dependencies) next to your config:

curl -O https://otpmock.com/sdk/otpmock.mjs

Then register a task that waits for the code:

// cypress.config.js
import { defineConfig } from "cypress";
import { OtpMock } from "./otpmock.mjs";

const otp = new OtpMock({ baseUrl: process.env.OTPMOCK_URL, apiKey: process.env.OTPMOCK_API_KEY });

export default defineConfig({
  e2e: {
    taskTimeout: 30000,
    setupNodeEvents(on) {
      on("task", {
        waitForCode: ({ phone, since }) => otp.waitForCode(phone, { since }).then((r) => r.code),
      });
    },
  },
});

waitForCode polls GET /v1/inbox/{phone}/code?since=... every 300 ms and resolves as soon as a code for that number shows up. A task must resolve to something serializable, so we return just the code string.

Step 3: the spec

// cypress/e2e/signup.cy.js
it("signs up with a phone number", () => {
  const phone = "+1555" + Cypress._.random(1000000, 9999999); // unique per test
  cy.visit("/signup");
  cy.get("[name=phone]").type(phone);

  const since = Date.now() - 5000; // margin for clock skew
  cy.contains("Send code").click();

  cy.task("waitForCode", { phone, since }).then((code) => {
    cy.get("[name=code]").type(code);
  });
  cy.contains("Verify").click();
  cy.contains("Welcome").should("be.visible");
});

Things that tripped me up

taskTimeout. The default is 60 s, which is fine, but if you lower it below your SMS wait time, the task fails with a confusing timeout. Keep it comfortably above the helper's own timeout (15 s by default).
Env vars in the wrong process. process.env in cypress.config.js is the Node side, so OTPMOCK_API_KEY must be in the environment that runs cypress run, not in Cypress.env().
Same number across retries. With retries enabled, a retried test reuses nothing from the failed attempt as long as the number is generated inside the test, and since makes sure an old code isn't picked up.
The app needs the variables too. The app under test sends the SMS, so OTPMOCK_URL has to be set where the app runs (staging, a preview deploy, a local server in CI), not just for Cypress.

Verify APIs work the same way

If your app uses Twilio Verify, Vonage Verify, Infobip 2FA or similar, the provider generates the code. The mock handles that too: it generates the code, puts it in the inbox for the task to read, and approves the check when the app submits it. The spec above doesn't change at all.

Wrapping up

I use otpmock for this (I built it after getting tired of the problem), but the cy.task pattern works with any mock inbox you can query over HTTP. The full Cypress guide, including CI setup, is here.

Next in the series: Selenium, with Python and Java.

How I test SMS OTP flows in Playwright without sending real texts

Kutay VURAL — Thu, 08 Oct 2026 19:11:24 +0000

Lately I've been working with OTP flows a lot. Every project seems to use a different SMS provider: Twilio here, Infobip there, AWS SNS somewhere else, and Netgsm or Verimor on the Turkish projects. Different APIs, different auth, different error formats.

But the part that annoyed me most wasn't the integration. It was the tests.

The end-to-end test runs fine until this screen:

Enter the code we sent to your phone.

And then it just sits there. The code went to a real phone, and the test has no way to read it.

The usual workarounds (and why I didn't like them)
Send a real SMS and check the phone. Slow, costs money on every run, and it isn't automation anymore if someone has to look at a phone.

Hard-code a backdoor. Something like "if the number is a test number, accept 000000". It works, but now you're not testing the real flow, and that if lives in your application code. One wrong environment variable and it's in production.

Skip the step. The most security-critical flow in the app ships untested.

None of these felt right, so I went looking for a cleaner pattern.

The pattern
In test and staging environments, the app still sends SMS through its usual provider SDK, but the SDK points to a mock inbox instead of the real provider.
Every test uses its own fake phone number, so parallel workers never read each other's codes.
Right before triggering the SMS, the test notes the time (since).
The test polls the inbox over HTTP for a code sent to that number after since, and types it in.
The app code doesn't change. Production keeps talking to the real provider. Nothing reaches a carrier in tests.

Step 1: point the SMS client at the mock (test env only)
With Twilio's Node SDK, you can pass a custom HTTP client that rewrites the host. This is the only change in the app, and it only kicks in when OTPMOCK_URL is set:

// src/sms.js
import twilio from "twilio";
import { OtpMockHttpClient } from "./twilio-node-client.mjs";

export const sms = twilio(process.env.TWILIO_ACCOUNT_SID, process.env.TWILIO_AUTH_TOKEN, {
  httpClient: process.env.OTPMOCK_URL ? new OtpMockHttpClient(process.env.OTPMOCK_URL) : undefined,
});

OtpMockHttpClient is about ten lines: it extends twilio.RequestClient and keeps the path, swapping only the host.

// twilio-node-client.mjs
import twilio from "twilio";

export class OtpMockHttpClient extends twilio.RequestClient {
  constructor(baseUrl = "https://api.otpmock.com") {
    super();
    this.baseUrl = baseUrl.replace(/\/+$/, "");
  }

  request(opts) {
    const u = new URL(opts.uri);
    return super.request({ ...opts, uri: this.baseUrl + u.pathname + u.search });
  }
}

Other SDKs are usually even simpler, because they already have a host option: Vonage has restHost/apiHost, the AWS SDK has endpoint, Infobip has baseUrl, Sinch has smsHostname.

In the test environment you also set the provider credential (here TWILIO_AUTH_TOKEN) to the mock's API key, so no real credentials are involved.

Step 2: a Playwright fixture with a unique number per test

// tests/fixtures.ts
import { test as base } from "@playwright/test";
import { OtpMock } from "./otpmock";

export const test = base.extend<{ otp: OtpMock; phone: string }>({
  otp: async ({}, use) => {
    await use(new OtpMock({ baseUrl: process.env.OTPMOCK_URL!, apiKey: process.env.OTPMOCK_API_KEY! }));
  },
  phone: async ({ otp }, use) => {
    const phone = otp.randomPhone(); // +1555XXXXXXX, unique per test
    await use(phone);
    await otp.clear(phone);          // optional tidy-up
  },
});

export { expect } from "@playwright/test";

Step 3: the test

// tests/signup.spec.ts
import { test, expect } from "./fixtures";

test("sign up with a phone number", async ({ page, otp, phone }) => {
  await page.goto("/signup");
  await page.getByLabel("Phone").fill(phone);

  const since = Date.now() - 5_000; // small margin for clock skew between app and runner
  await page.getByRole("button", { name: "Send code" }).click();

  const { code } = await otp.waitForCode(phone, { since });
  await page.getByLabel("Code").fill(code);
  await page.getByRole("button", { name: "Verify" }).click();

  await expect(page.getByText("Welcome")).toBeVisible();
});

waitForCode polls GET /v1/inbox/{phone}/code?since=... every 300 ms and returns as soon as the code arrives. In practice that's well under a second after the app's send call.

What about Verify APIs?
This is where hard-coded backdoors really fall apart. With Twilio Verify (or Vonage Verify, Infobip 2FA, etc.) the provider generates the code, and your app only calls verifications.create and later verificationChecks.create.

A mock that understands those APIs handles both halves: it generates the code, drops it in the inbox for your test to read, and approves the check when your app submits the right code. Your app code stays exactly the same, and wrong-code and too-many-attempts errors behave like the real thing, so you can test those screens too.

Running it in CI
Set the variables wherever both the app under test and the test runner can read them. A mistake I made early on: setting them only for the test runner. The app is the one sending the SMS, so it needs OTPMOCK_URL too.

# .github/workflows/e2e.yml
env:
  OTPMOCK_URL: https://api.otpmock.com
  OTPMOCK_API_KEY: ${{ secrets.OTPMOCK_API_KEY }}
  TWILIO_AUTH_TOKEN: ${{ secrets.OTPMOCK_API_KEY }}   # provider credential = mock key in tests

A few pitfalls
No since. Without it, a retried test can pick up the code from its previous attempt.
Number formatting. If your UI adds a country code or strips the +, read the code for the number the app actually sent to.
Fixed sleeps. Poll instead of waitForTimeout(5000). It's both faster and less flaky.

Not using Playwright?
The pattern is the same everywhere: unique number per test, take since, poll the inbox over HTTP, type the code. Only the way you make the HTTP call changes. I wrote up the details for other tools:

Cypress (read the code from a cy.task)
Selenium (Python and Java)
WebdriverIO, Puppeteer, TestCafe
Mobile: Appium, Maestro, Detox for React Native
API and QA tools: Postman/Newman, Robot Framework, k6 load tests

The tool
I ended up turning my mock into a small hosted service called otpmock. It currently emulates 15 SMS providers (Twilio, Vonage, AWS SNS, Infobip, Sinch, Telnyx, Plivo, Netgsm and others) plus their Verify APIs, and the setup is tested with the official SDKs in Node, Python, Java, C#, PHP, Ruby and Go. There's a free tier if you want to try the pattern above without building the mock yourself.

But the pattern is the important part. Whether you use otpmock, WireMock or something you write yourself, the OTP step doesn't have to be the one part of your login flow that never gets tested.

How do you handle OTP steps in your E2E tests? I'm curious what other people do.