DEV Community: Willette

Firebase Phone Auth Testing in Flutter: Test Numbers vs Real SMS

Willette — Thu, 12 Nov 2026 10:55:06 +0000

Disclosure: I work with HeroSMS, the virtual number provider used in the examples below. Written with AI tools; code and endpoints checked against the HeroSMS API reference.

If you've shipped Firebase phone auth in a Flutter app, this story may sound familiar. On your machine everything is green: you add a test number in the Firebase console, type 123456, sign in, merge. After the release, the reports start: "I never got the code." Sometimes it's a whole country, sometimes a few users. The logs are empty, because verifyPhoneNumber returns Future<void> and the real error sits in a verificationFailed callback nobody logs.

Firebase test phone numbers (the docs call them fictional numbers, and so will I) send no SMS. They prove your app calls the API correctly, not that a real SMS reaches a real phone in each country. Real-SMS testing has its own problems: emulators can't use real numbers, CI runners have no SIM, and every run sends a billable SMS.

This post splits Firebase phone auth testing into two layers: fictional numbers on every PR, and a scheduled smoke test that sends a real SMS to a number rented in each market. Stack: Flutter, firebase_auth, integration_test, GitHub Actions.

Set up the Firebase project for phone auth

Many "OTP not sent" bugs come from project config, not Dart code:

Link Cloud Billing (Blaze). Since September 2024, a project needs Cloud Billing to use the SMS service.
Enable the Phone provider under Security > Authentication > Sign-in method.
Set an SMS region policy under Security > Authentication > Settings. For new projects, the default policy allows no regions. Use an allowlist or a denylist; only one can be active.
Android: add SHA-1 and SHA-256 fingerprints (SHA-256 is for Play Integrity). Without Google Play services, or when the app isn't distributed through the Play Store, the SDK falls back to reCAPTCHA, which needs SHA-1 and an API key that is unrestricted or allowlisted for PROJECT_ID.firebaseapp.com.
iOS: enable Push Notifications and Background Modes (Background fetch, Remote notifications), upload an APNs key, and add your Encoded App ID as a URL scheme for the reCAPTCHA fallback.
Web: add your domain to Authorized domains. Per the docs, localhost is not allowed as a hosted domain for phone auth.

Packages: firebase_auth 7.x (7.0.1 as I write; 7.0 requires Android minSdk 24) and firebase_core 5.x; integration_test, flutter_test and http as dev dependencies.

verifyPhoneNumber and its four callbacks

verifyPhoneNumber returns Future<void>. Results arrive through four callbacks:

verificationCompleted: Android only. The SMS was read automatically, or the number was verified instantly without a code.
verificationFailed: bad number format, blocked region, quota exceeded.
codeSent(verificationId, resendToken): the SMS is on its way. On iOS, resendToken is always null.
codeAutoRetrievalTimeout: Android stopped waiting for auto-retrieval (timeout defaults to 30 seconds).

Numbers need a + and a country code. Web uses signInWithPhoneNumber, which returns a ConfirmationResult and renders an invisible reCAPTCHA by default. One service wraps both paths, so the UI and the tests share it:

// lib/phone_auth_service.dart
import 'dart:async';

import 'package:firebase_auth/firebase_auth.dart';
import 'package:flutter/foundation.dart' show debugPrint, kIsWeb;

class PhoneAuthService {
  PhoneAuthService([FirebaseAuth? auth]) : _auth = auth ?? FirebaseAuth.instance;

  final FirebaseAuth _auth;
  String? _verificationId;
  int? _resendToken; // Android only; always null on iOS
  ConfirmationResult? _webResult; // web only

  /// UserCredential: Android finished verification on its own (instant or auto-read SMS).
  /// null: the SMS was sent; wait for the user to type the code, then call confirm().
  /// Error: the FirebaseAuthException from verificationFailed.
  Future<UserCredential?> start(String phoneE164, {bool resend = false}) async {
    if (kIsWeb) {
      // Without a RecaptchaVerifier, the SDK creates an invisible reCAPTCHA
      _webResult = await _auth.signInWithPhoneNumber(phoneE164);
      return null;
    }
    final done = Completer<UserCredential?>();
    await _auth.verifyPhoneNumber(
      phoneNumber: phoneE164, // E.164 like +84...; the country code is required
      timeout: const Duration(seconds: 60), // default 30 seconds, max 2 minutes
      forceResendingToken: resend ? _resendToken : null,
      verificationCompleted: (PhoneAuthCredential credential) async {
        // Android only. It can arrive AFTER codeSent and sign the user in silently,
        // so the UI should listen to authStateChanges() instead of only awaiting this Future.
        try {
          final uc = await _auth.signInWithCredential(credential);
          if (!done.isCompleted) done.complete(uc);
        } on FirebaseAuthException catch (e) {
          if (!done.isCompleted) done.completeError(e);
        }
      },
      verificationFailed: (FirebaseAuthException e) {
        debugPrint('verificationFailed: ${e.code}'); // the real error lives here, don't swallow it
        if (!done.isCompleted) done.completeError(e);
      },
      codeSent: (String verificationId, int? resendToken) {
        _verificationId = verificationId;
        _resendToken = resendToken;
        if (!done.isCompleted) done.complete(null);
      },
      codeAutoRetrievalTimeout: (String verificationId) {
        _verificationId = verificationId; // Android only
      },
    );
    return done.future;
  }

  Future<UserCredential> confirm(String smsCode) {
    if (kIsWeb) return _webResult!.confirm(smsCode);
    final credential = PhoneAuthProvider.credential(
      verificationId: _verificationId!,
      smsCode: smsCode,
    );
    return _auth.signInWithCredential(credential);
  }
}

/// The docs only show 'invalid-phone-number'; the other code is a common example,
/// so always keep the default branch.
String phoneAuthMessage(FirebaseAuthException e) => switch (e.code) {
      'invalid-phone-number' => 'Invalid phone number. Include the country code.',
      'too-many-requests' => 'Too many attempts. Please wait a moment.',
      _ => 'Could not send the code (${e.code}). Please try again later.',
    };

Two easy-to-miss details:

Calling verifyPhoneNumber again for the same number before the previous request times out sends no new SMS, so "Resend code" should call start(phone, resend: true) to pass forceResendingToken (Android only).
Google stores the phone numbers you collect to fight abuse, so get consent and tell users an SMS is coming.

Firebase test phone numbers: test the OTP flow without SMS

A fictional number is a phone number paired with a fixed 6-digit code, added under Security > Authentication > Sign-in method > Phone > Phone numbers for testing (up to 10). No SMS is sent, no quota is used, and repeated runs aren't throttled. The Flutter phone auth docs say phone sign-in only works on real devices and the web; on emulators, use fictional numbers.

Pick fake but well-formed numbers, such as the US 555 range (+1 650-555-3434 in the web guide). A fictional user's ID token has the same signature as a real one, so use hard-to-guess numbers and codes, rotate them, and keep them out of production builds.

setSettings(appVerificationDisabledForTesting: true) turns off app verification (Play Integrity, APNs, reCAPTCHA), so tests run on emulators and simulators. It works only with fictional numbers: with the flag on, a real number fails to sign in. Call it before verifyPhoneNumber, from test code only.

// integration_test/phone_auth_test.dart
// Run: flutter test integration_test/phone_auth_test.dart -d emulator-5554 \
//   --dart-define-from-file=test_secrets.json   (this file is in .gitignore)
import 'package:firebase_auth/firebase_auth.dart';
import 'package:firebase_core/firebase_core.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:integration_test/integration_test.dart';
import 'package:my_app/firebase_options.dart'; // generated by `flutterfire configure`
import 'package:my_app/phone_auth_service.dart';

// Must be const, or String.fromEnvironment isn't guaranteed to read the value
const testPhone = String.fromEnvironment('TEST_PHONE');
const testCode = String.fromEnvironment('TEST_SMS_CODE');

void main() {
  IntegrationTestWidgetsFlutterBinding.ensureInitialized();

  setUpAll(() async {
    await Firebase.initializeApp(options: DefaultFirebaseOptions.currentPlatform);
    // ONLY affects numbers listed under "Phone numbers for testing"
    await FirebaseAuth.instance.setSettings(appVerificationDisabledForTesting: true);
  });

  tearDown(() => FirebaseAuth.instance.signOut());

  testWidgets('signs in with a Firebase fictional number', (tester) async {
    expect(testPhone, isNotEmpty, reason: 'TEST_PHONE is missing from --dart-define');
    expect(testCode, isNotEmpty, reason: 'TEST_SMS_CODE is missing from --dart-define');
    final service = PhoneAuthService();
    // Bounded, so a CI job doesn't hang if no callback ever fires
    final auto = await service.start(testPhone).timeout(const Duration(seconds: 90));
    final uc = auto ?? await service.confirm(testCode);
    expect(uc.user, isNotNull);
    expect(FirebaseAuth.instance.currentUser?.uid, uc.user!.uid);
  });
}

To test auto-retrieval, also call setSettings(phoneNumber: testPhone, smsCode: testCode) before start(), and verificationCompleted fires right away (Android only). On web, integration tests need flutter drive and the localhost rule gets in the way, so this post sticks to Android and iOS.

Why Firebase OTP still fails in production

The test above sends no SMS and skips app verification, so it can't tell you whether a real SMS reaches users in each country. Common causes, from the Firebase Auth FAQ and the phone auth guides (error codes are real-world examples, not an official list):

Symptom	Cause	Fix
SMS service can't be enabled	No Cloud Billing	Upgrade to Blaze
New project, real numbers get nothing	Default region policy allows no regions	Add your markets to the allowlist
`operation-not-allowed`, "SMS unable to be sent until this region enabled by the app developer"	Region blocked by the policy (from community reports, not the docs)	Check the allowlist or denylist
`too-many-requests`	Too many requests from one device, or SMS quota used up	Wait, then retry; don't hammer
App verification errors	Missing SHA, restricted API key, missing APNs key or URL scheme	See the setup list
`codeSent` fires but no SMS arrives	Carrier didn't deliver; number was ported to another carrier	Try another number

The FAQ says plainly that not every carrier delivers verification SMS reliably. Ported numbers currently get no SMS at all, with no workaround, so ask users who never got a code whether they switched carriers. Firebase also limits how many SMS go to one number (the figure isn't published), so don't point every test run at the same number.

The most direct way to catch these before your users do is a real SMS to a number in each market ahead of release.

Test Firebase SMS OTP with a real number per market

Rent a temporary number in the country you're checking, let Firebase send a real SMS to it, read the code over an API, and sign in. I use HeroSMS as the example; with another provider, you only rewrite SmsProvider.

The client speaks the handler_api protocol many virtual-number services share: GET requests to https://hero-sms.com/stubs/handler_api.php with api_key and action in the query string. HeroSMS also has a REST API at https://hero-sms.com/api/v1 with an Authorization: ApiKey <token> header, which keeps the key out of URLs; the lifecycle is the same, so switching only touches SmsProvider. Three actions:

getNumberV2 rents a number: JSON with activationId, phoneNumber (no +), activationCost, activationEndTime.
getStatus returns STATUS_WAIT_CODE while waiting, STATUS_OK:<code> once the code is in, STATUS_CANCEL if canceled.
setStatus with 6 completes the activation; 8 cancels it and refunds the price.

Errors are JSON like {"title": "BAD_KEY", ...} with an HTTP 4xx, so check the status code first. NO_NUMBERS is a plain string with HTTP 200. setStatus can return 409, for example EARLY_CANCEL_DENIED (no cancel in the first 2 minutes) or OTP_RECEIVED (the SMS already arrived).

The catch: an integration test runs on the phone, not on the CI machine, so it can't read the runner's environment. The URL and key come in through --dart-define.

// integration_test/support/sms_provider.dart
import 'dart:async';
import 'dart:convert';

import 'package:http/http.dart' as http;

// The test runs on the phone and can't see the runner's env: values come in via --dart-define
const _apiUrl = String.fromEnvironment('SMS_API_URL'); // the .../stubs/handler_api.php endpoint
const _apiKey = String.fromEnvironment('SMS_API_KEY'); // CI secret, never committed

class SmsProviderException implements Exception {
  SmsProviderException(this.code, [this.httpStatus]);
  final String code; // BAD_KEY, NO_NUMBERS, NO_BALANCE, EARLY_CANCEL_DENIED...
  final int? httpStatus;
  @override
  String toString() => 'SmsProviderException($code, HTTP ${httpStatus ?? 200})';
}

class Activation {
  Activation(this.id, this.phone, this.cost) : rentedAt = DateTime.now();
  final String id;
  final String phone; // +E.164
  final String cost; // activationCost: add these up to see what each run costs
  final DateTime rentedAt;
}

class SmsProvider {
  final _http = http.Client();

  Future<Object?> _call(Map<String, String> params) async {
    if (_apiUrl.isEmpty || _apiKey.isEmpty) {
      throw StateError('SMS_API_URL or SMS_API_KEY is missing'); // don't print the values
    }
    final base = Uri.parse(_apiUrl);
    final uri = base.replace(
        queryParameters: {...base.queryParameters, 'api_key': _apiKey, ...params});
    final http.Response res;
    try {
      res = await _http.get(uri).timeout(const Duration(seconds: 30));
    } on http.ClientException {
      throw SmsProviderException('NETWORK'); // the original message contains the URI, api_key included
    }
    Object? body;
    try {
      body = jsonDecode(res.body); // a JSON object or a quoted string
    } on FormatException {
      body = res.body.trim(); // plain text: STATUS_OK:123456, NO_NUMBERS...
    }
    if (res.statusCode >= 400) {
      // Errors are JSON {"title":"BAD_KEY","details":"..."} with HTTP 4xx/5xx
      final code = body is Map ? '${body['title']}' : 'HTTP_${res.statusCode}';
      throw SmsProviderException(code, res.statusCode);
    }
    return body;
  }

  /// service: code from action=getServicesList; country: ID from action=getCountries
  Future<Activation> rentNumber({required String service, required int country}) async {
    final body = await _call(
        {'action': 'getNumberV2', 'service': service, 'country': '$country'});
    if (body is! Map) throw SmsProviderException('$body'); // NO_NUMBERS: a string, HTTP 200
    return Activation(
        '${body['activationId']}', '+${body['phoneNumber']}', '${body['activationCost']}');
  }

  /// Polls getStatus until STATUS_OK:<code>; keeps waiting on STATUS_WAIT_*.
  Future<String> waitForCode(String id, {Duration timeout = const Duration(minutes: 3)}) async {
    final deadline = DateTime.now().add(timeout);
    while (DateTime.now().isBefore(deadline)) {
      final status = '${await _call({'action': 'getStatus', 'id': id})}';
      if (status.startsWith('STATUS_OK:')) return status.substring('STATUS_OK:'.length);
      // STATUS_CANCEL or anything unexpected: stop now instead of polling until the timeout
      if (!status.startsWith('STATUS_WAIT')) throw SmsProviderException(status);
      await Future<void>.delayed(const Duration(seconds: 5));
    }
    throw TimeoutException('No SMS for activation $id', timeout);
  }

  /// 6 = complete (code used), 8 = cancel and refund (code not used)
  Future<void> close(Activation a, {required bool used}) async {
    Future<Object?> setStatus(String s) =>
        _call({'action': 'setStatus', 'id': a.id, 'status': s});
    try {
      await setStatus(used ? '6' : '8');
    } on SmsProviderException catch (e) {
      if (e.code == 'EARLY_CANCEL_DENIED') {
        // No cancel in the first 2 minutes (minActivationTime: 120): wait past that mark, then cancel
        final wait = a.rentedAt.add(const Duration(seconds: 125)).difference(DateTime.now());
        if (!wait.isNegative) await Future<void>.delayed(wait);
        await setStatus('8');
      } else if (e.code == 'OTP_RECEIVED') {
        await setStatus('6'); // the SMS arrived: close the activation as completed
      } else {
        rethrow;
      }
    }
  }

  void dispose() => _http.close();
}

The test reuses PhoneAuthService:

// integration_test/phone_auth_real_test.dart (real devices only)
import 'package:firebase_auth/firebase_auth.dart';
import 'package:firebase_core/firebase_core.dart';
import 'package:flutter/foundation.dart' show debugPrint;
import 'package:flutter_test/flutter_test.dart';
import 'package:integration_test/integration_test.dart';
import 'package:my_app/firebase_options.dart'; // points to the Firebase project used for testing
import 'package:my_app/phone_auth_service.dart';

import 'support/sms_provider.dart';

const otpService = String.fromEnvironment('OTP_SERVICE'); // code from getServicesList
const otpCountry = int.fromEnvironment('OTP_COUNTRY', defaultValue: -1); // ID from getCountries

void main() {
  IntegrationTestWidgetsFlutterBinding.ensureInitialized();

  // Do NOT enable appVerificationDisabledForTesting: real numbers would fail to sign in
  setUpAll(() => Firebase.initializeApp(options: DefaultFirebaseOptions.currentPlatform));

  testWidgets('a real SMS OTP reaches the rented number and signs in', (tester) async {
    // Check the config before spending money (never put the key in expect)
    expect(otpCountry, isNonNegative, reason: '--dart-define=OTP_COUNTRY is missing');

    final sms = SmsProvider();
    Activation? activation;
    var used = false;
    try {
      final a = await sms.rentNumber(service: otpService, country: otpCountry);
      activation = a;
      debugPrint('activationCost: ${a.cost}');
      final auth = PhoneAuthService();
      // Bounded: if it gets stuck on reCAPTCHA, the finally block still cancels the number
      final auto = await auth.start(a.phone).timeout(const Duration(minutes: 2));
      if (auto != null) {
        // Android verified instantly, no SMS: this run can't check anything
        await auto.user?.delete();
        fail('instant verification, no SMS to check');
      }
      final code = await sms.waitForCode(a.id);
      used = true; // the SMS arrived: close with status 6 even if a later step fails
      final uc = await auth.confirm(code);
      expect(uc.user?.phoneNumber, isNotNull);
      await uc.user?.delete(); // remove the test user to keep the project clean
    } finally {
      if (activation != null) {
        try {
          await sms.close(activation, used: used);
        } catch (e) {
          debugPrint('Could not close the activation: $e'); // error code only, no key
        }
      }
      sms.dispose();
      await FirebaseAuth.instance.signOut();
    }
  },
      // Without OTP_SERVICE the test is skipped, so a normal pipeline running the whole folder never rents a number
      skip: otpService.isEmpty,
      timeout: const Timeout(Duration(minutes: 8)));
}

Things to watch:

With OTP_SERVICE empty the test skips itself, so a regular flutter test integration_test run never rents a number. It needs a real Android phone with Google Play services or a real iPhone.
A build installed by flutter test isn't distributed through the Play Store, so Android uses reCAPTCHA, not Play Integrity. If a challenge appears and nobody solves it, start() times out after 2 minutes and the number is canceled. Keep a person near the device to solve it by hand; never bypass it automatically. Check Play Integrity with a build from an internal testing track.
The 5-second poll interval is my choice, not a documented recommendation.

Honest limits:

An activation lasts about 20 minutes (exact end in activationEndTime); the 3-minute wait fits easily.
A number that gets no SMS is canceled with status 8 and the price returns to your balance. One that got the SMS is billed, even if your app fails afterwards.
Some services reject virtual numbers, and the rented number's carrier may differ from your users'. It's a smoke test, not a guarantee for every SIM.
Every run costs one Firebase SMS plus one number, so schedule it instead of running it per commit.

Run the real-SMS test on a schedule in CI

The fictional-number test runs on an emulator in your normal pipeline. The real-number test needs a real phone, so it lives on a self-hosted runner with an Android device plugged in, triggered weekly or by hand before a release:

# .github/workflows/phone-auth-real-sms.yml
name: phone-auth-real-sms

on:
  workflow_dispatch:            # run by hand before a release
  schedule:
    - cron: '0 1 * * 1'         # 01:00 UTC on Monday = 08:00 in Vietnam

concurrency:
  group: phone-auth-real-sms    # one phone: never let two runs overlap
  cancel-in-progress: false

jobs:
  real-sms:
    runs-on: [self-hosted, android-device]   # runner with Flutter installed and a real Android phone plugged in
    strategy:
      fail-fast: false
      max-parallel: 1                          # one device: one country at a time
      matrix:
        country: [VN, ID]                      # every country must be allowed by the region policy
    steps:
      - uses: actions/checkout@v7
      - run: flutter pub get
      - name: Integration test with a real number (${{ matrix.country }})
        env:
          SMS_API_URL: ${{ vars.SMS_API_URL }}
          SMS_API_KEY: ${{ secrets.SMS_API_KEY }}   # Secrets only
          OTP_SERVICE: ${{ vars.OTP_SERVICE }}
          OTP_COUNTRY: ${{ vars[format('OTP_COUNTRY_{0}', matrix.country)] }}
          DEVICE_ID: ${{ vars.ANDROID_DEVICE_ID }}
        run: |
          test -n "$OTP_SERVICE" || { echo 'OTP_SERVICE is not configured'; exit 1; }  # if it's missing, the test is skipped and the job still goes green
          flutter test integration_test/phone_auth_real_test.dart -d "$DEVICE_ID" \
            --dart-define=SMS_API_URL="$SMS_API_URL" \
            --dart-define=SMS_API_KEY="$SMS_API_KEY" \
            --dart-define=OTP_SERVICE="$OTP_SERVICE" \
            --dart-define=OTP_COUNTRY="$OTP_COUNTRY"

DEVICE_ID is the phone's serial; each market gets an OTP_COUNTRY_<code> repository variable with its ID from getCountries. OTP_SERVICE must be the service code for your Firebase SMS sender, since an activation is reserved for one service. Swap [VN, ID] for your markets, and pick a cron time when someone is near the phone in case a reCAPTCHA challenge appears.

What a Firebase OTP test run costs

Firebase bills every phone auth SMS by destination country (Identity Platform pricing). Prices on November 12, 2026 (recheck, they change):

Country	Price per SMS
Vietnam	$0.13
Indonesia	$0.35
India	$0.07
Türkiye	$0.01
United States	$0.01

The same page says the first ten SMS per day are not billed. That isn't a Spark free tier: the project still needs billing.
Per the limits page, Firebase Authentication sends up to 3,000 SMS per day (no limit with Identity Platform), and each IP address is capped at 50 SMS per minute and 500 per hour.
Every "Resend code" tap is another SMS. The rented number is billed separately; the test prints its activationCost.

Security and responsible use

Keep the key out of the repo and logs. Use --dart-define-from-file with a git-ignored file locally and Secrets in CI. Don't log SMS text or raw http exceptions (their message contains the URI, api_key included).
The test build contains the key. --dart-define values are compiled into the binary. Don't upload that build as an artifact, keep a small balance, and rotate the key if it might have leaked.
Use it on your own app only. Rented numbers are for test accounts in apps you own or are authorized to test. Don't use them to mass-register accounts, to get around another service's verification, or for banking and identity-bound services. Don't test with colleagues' or customers' numbers either.
Never run this against production. Use a separate test project with the same billing, region policy and app verification setup.

Protect production from SMS pumping

SMS pumping (toll fraud) is bots hitting "Send code" in bulk for number ranges that pay the attacker a cut, while you pay the bill. What helps:

Region policy: allowlist only the markets you serve.
Watch verified/sent per region. The FAQ calls 70 to 85% normal; below 50% is a common sign of bots or pumping.
Remove localhost from Authorized domains in production (projects created after April 28, 2025 don't include it).
App Check for Firebase Authentication is in Preview, requires Identity Platform, and doesn't stop flows started and abandoned just to generate SMS.
reCAPTCHA SMS defense (also Preview) scores each SMS request for toll-fraud risk and, in enforce mode, sends nothing above your threshold. It needs the reCAPTCHA Enterprise API and Android Auth 23.1.0+ or iOS 11.6.0+.

FAQ

Is Firebase phone auth free?
Not really. Since September 2024 you need Cloud Billing (Blaze) to send SMS. The first ten SMS per day aren't billed; after that you pay per SMS, by country. Fictional numbers cost nothing.

Can I test Firebase SMS OTP on the Android emulator?
Yes, with a fictional number and appVerificationDisabledForTesting: true. Real numbers only work on real devices and the web.

Wrap-up

Test Firebase phone auth in two layers: fictional numbers on every PR, and real temporary numbers per market before release. Only the second sends a real SMS with app verification on. Pre-release checklist:

Billing is linked; the region policy covers every target market and nothing extra.
SHA-1/SHA-256 for every signing key (including the Play App Signing key from Play Console), the APNs key and Authorized domains are in place.
appVerificationDisabledForTesting and fictional numbers live only in test code; the provider key lives in secrets.
After release, you track verified/sent per region.

How do you test Firebase phone auth today: fictional numbers only, a shared phone in a drawer, or something else?

How to Handle OTP in Selenium Java with a Real SMS Number API

Willette — Wed, 04 Nov 2026 20:54:04 +0000

Disclosure: I work with HeroSMS, the virtual number provider used in the examples below. Written with AI tools; code and endpoints checked against the HeroSMS API reference.

If you've tried to handle OTP in Selenium WebDriver, you know this test. It types a phone number, clicks "Send code", then sits in Thread.sleep(30000) while you read the code off your phone and type it in. That works on your own machine. In the nightly run nobody types anything, so the next explicit wait fails:

org.openqa.selenium.TimeoutException: Expected condition failed: waiting for url to contain "/welcome". Current url: "https://staging.example.com/signup/verify"
(tried for 15 seconds with 500 milliseconds interval)

The CI runner has no SIM card. This tutorial replaces the human with an API. The test rents a real number, polls for the SMS code from Java with a deadline, types the code after an explicit wait and releases the number in finally. GitHub Actions runs the whole thing on a schedule. Stack: Java 17+, Maven, Selenium 4.52 and JUnit 5.

How to automate OTP in Selenium WebDriver

Selenium can't read SMS because it drives a browser, not a phone. To automate OTP in Selenium:

Rent a phone number through an SMS API and type it into your form.
Poll the API from Java until the code arrives.
Wait for the OTP field with WebDriverWait and type the code into it.

Your test code does the polling, and Selenium only does the typing. The same flow works for sign-up, login and SMS 2FA.

Mocks, test numbers, 2FA off for test users, or a real number

Start with the free options, since real numbers cost money. Selenium's own documentation advises against automating 2FA at all. It suggests these workarounds instead:

disable 2FA for test users or in the test environment
accept a test-only token
skip 2FA for requests from your test machines' IPs

Option	What it proves	Cost	When to run
Mocked SMS sender	UI and app logic	Free	Every PR
Provider test credentials or fictional numbers	Your call to the SMS provider; UI with a fixed code	Free	Every PR
2FA off for test users, or a test-only token	Everything after the second factor	Free	Every PR
TOTP secret in the test	Your authenticator-app step	Free	Every PR
Real rented number	A real SMS reaches a real number and your app accepts the code	Paid per number	Nightly, pre-release

Twilio's test credentials don't connect to real phone numbers. Firebase's fictional test numbers (up to 10, each with a fixed code) send no SMS. If your second factor is an authenticator app, you don't need a phone: generate the TOTP code from the shared secret.

None of these options proves that an SMS actually reaches a phone in your launch country. A real-number test fills that gap. I built the same split (mocks on every PR, real SMS on a schedule) three times before: in Python, for Playwright with a webhook and for Postman and Newman. This post is the Selenium Java version.

What you need

Java 17 or later (the CI job below uses 21), Maven 3.6.3 or later, and Chrome. Selenium Manager has been built in since 4.6 and downloads and caches the matching driver. Since 4.11 it can also fetch Chrome for Testing if no browser is installed.
A HeroSMS API key and some balance. What you buy is an activation: one number reserved for one service in one country for about 20 minutes. The exact end time is in expiredAt. Numbers cover 180+ countries, so you can usually test in the country you launch in.
These environment variables:

export HEROSMS_API_KEY="<your API key>"
export OTP_SERVICE="<service code>"   # 2-4 letters
export OTP_COUNTRY="<country ID>"     # an integer
export APP_URL="https://staging.example.com"
# optional: export HEROSMS_MAX_PRICE="<price cap per number>"

Service codes and country IDs come from the getServicesList and getCountries endpoints in the HeroSMS API reference. GET /activations/offers/sms without filters returns them too: service codes are the keys of data, and country IDs are the keys inside each service.

The project has four classes under src/test and a workflow:

selenium-otp/
├── pom.xml
├── .github/workflows/otp-nightly.yml
└── src/test/java/com/example/otp/
    ├── HeroSmsClient.java
    ├── SmsCodePoller.java
    ├── ActivationCleanup.java
    └── OtpSignupTest.java

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.example</groupId>
  <artifactId>selenium-otp</artifactId>
  <version>1.0-SNAPSHOT</version>

  <properties>
    <maven.compiler.release>17</maven.compiler.release>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>org.junit</groupId>
        <artifactId>junit-bom</artifactId>
        <version>5.14.4</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

  <dependencies>
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>4.52.0</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-databind</artifactId>
      <version>2.22.4</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter</artifactId>
      <scope>test</scope>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.16.0</version>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>3.6.1</version>
      </plugin>
    </plugins>
  </build>
</project>

To use JUnit 6 (which needs Java 17+), change the BOM version to 6.1.3. The test compiles unchanged because it only uses the Jupiter API. HTTP calls go through the JDK's own HttpClient, so Jackson is the only extra dependency.

Step 1: a small SMS API client in Java

// src/test/java/com/example/otp/HeroSmsClient.java
package com.example.otp;

import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;

import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.time.Duration;
import java.util.LinkedHashMap;
import java.util.Map;

/** Minimal HeroSMS v1 client: rent a number, read the last OTP, finish or cancel. */
public final class HeroSmsClient {
    private static final String API = "https://hero-sms.com/api/v1";
    private static final ObjectMapper JSON = new ObjectMapper();

    private final HttpClient http = HttpClient.newHttpClient(); // immutable, reuse it
    private final String apiKey;

    public HeroSmsClient(String apiKey) {
        if (apiKey == null || apiKey.isBlank()) {
            throw new IllegalStateException("HEROSMS_API_KEY is not set");
        }
        this.apiKey = apiKey;
    }

    /** One rented number. phone is digits without the '+'. */
    public record Activation(String id, String phone, String countryPhoneCode, String expiredAt) {}

    /** Error body from the API: branch on title, not on the HTTP status. */
    public static final class HeroSmsException extends IOException {
        public final int status;
        public final String title;
        public final String details;
        public final transient JsonNode info;
        public final long retryAfter; // seconds, 0 if the header is absent

        HeroSmsException(int status, String title, String details, JsonNode info, long retryAfter) {
            super("HTTP " + status + " " + title + ": " + details);
            this.status = status;
            this.title = title;
            this.details = details;
            this.info = info;
            this.retryAfter = retryAfter;
        }
    }

    public Activation buy(String service, int country, Double maxPrice)
            throws IOException, InterruptedException {
        Map<String, Object> body = new LinkedHashMap<>();
        body.put("service", service);
        body.put("country", country);
        body.put("amount", 1);
        body.put("verificationType", "sms");
        if (maxPrice != null) body.put("maxPrice", maxPrice);

        JsonNode a = call("POST", "/activations", body).path("data").path(0);
        if (a.isMissingNode()) throw new IOException("POST /activations returned no activation");
        return new Activation(a.path("id").asText(), a.path("phone").asText(),
                a.path("countryPhoneCode").asText(), a.path("expiredAt").asText());
    }

    /** data of GET /activations/{id}/otp/last: smsCode and smsText, either may be null. */
    public JsonNode lastOtp(String id) throws IOException, InterruptedException {
        return call("GET", "/activations/" + id + "/otp/last", null).path("data");
    }

    /** Closes the activation after the code was used. No refund. */
    public void finish(String id) throws IOException, InterruptedException {
        call("POST", "/activations/" + id + "/finish", null);
    }

    /** Cancels an activation that got no SMS. The price is refunded. */
    public void cancel(String id) throws IOException, InterruptedException {
        call("DELETE", "/activations/" + id, null);
    }

    private JsonNode call(String method, String path, Object body)
            throws IOException, InterruptedException {
        HttpRequest.Builder req = HttpRequest.newBuilder(URI.create(API + path))
                .timeout(Duration.ofSeconds(15)) // without it, send() can block forever
                .header("Authorization", "ApiKey " + apiKey)
                .header("Accept", "application/json");
        if (body != null) {
            req.header("Content-Type", "application/json")
               .method(method, HttpRequest.BodyPublishers.ofString(JSON.writeValueAsString(body)));
        } else {
            req.method(method, HttpRequest.BodyPublishers.noBody());
        }

        HttpResponse<String> res = http.send(req.build(), HttpResponse.BodyHandlers.ofString());
        String text = res.body() == null ? "" : res.body();
        if (res.statusCode() >= 400) {
            JsonNode err = parse(text);
            long retryAfter = res.headers().firstValue("Retry-After")
                    .map(HeroSmsClient::seconds).orElse(0L);
            throw new HeroSmsException(res.statusCode(), err.path("title").asText(""),
                    err.path("details").asText(""), err.path("info"), retryAfter);
        }
        return parse(text); // 204 No Content -> missing node
    }

    private static JsonNode parse(String text) {
        if (text.isBlank()) return JSON.missingNode();
        try {
            return JSON.readTree(text);
        } catch (IOException e) {
            return JSON.missingNode(); // not JSON
        }
    }

    private static long seconds(String value) {
        try {
            return Long.parseLong(value.trim());
        } catch (NumberFormatException e) {
            return 0; // an HTTP date: use the caller's fallback
        }
    }
}

Auth is one header, Authorization: ApiKey <key>. The key is read from the environment and never touches the browser.
Errors come back as JSON with title and details, and sometimes info. HeroSmsException keeps all three plus the Retry-After header, so callers can branch on title.
Always set HttpRequest.timeout. Without it, send() can wait forever, and one stuck request hangs the whole job. With it, a slow request throws an HttpTimeoutException.
phone comes back as an integer without the +. asText() keeps every digit.

Step 2: poll for the code with a deadline, not Thread.sleep

A fixed Thread.sleep(30000) is the classic way to wait for an OTP, and it fails in both directions. It's too short when a carrier is slow, and it wastes 28 seconds when the SMS lands in two. Poll instead, and stop at a deadline:

// src/test/java/com/example/otp/SmsCodePoller.java
package com.example.otp;

import com.example.otp.HeroSmsClient.HeroSmsException;
import com.fasterxml.jackson.databind.JsonNode;

import java.io.IOException;
import java.time.Duration;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

/** Polls GET /activations/{id}/otp/last until a code arrives or the deadline passes. */
public final class SmsCodePoller {
    private static final Set<String> CLOSED =
            Set.of("ACTIVATION_NOT_ACTIVE", "CANCELED", "FINISHED", "REFUNDED");
    private static final Pattern CODE = Pattern.compile("\\b\\d{4,8}\\b"); // fit it to your template

    private SmsCodePoller() {}

    public static String waitForCode(HeroSmsClient sms, String activationId, Duration timeout)
            throws IOException, InterruptedException {
        long deadline = System.nanoTime() + timeout.toNanos();
        long delayMs = 2_000; // backs off to 10 s
        IOException lastError = null; // becomes the cause if time runs out
        while (System.nanoTime() < deadline) {
            long sleepMs = delayMs;
            try {
                JsonNode otp = sms.lastOtp(activationId);
                lastError = null; // the API answered
                String code = extractCode(otp);
                if (code != null) return code;
                if (!text(otp.path("smsText")).isEmpty()) {
                    throw new IllegalStateException("SMS arrived without a 4-8 digit code: adjust the regex");
                }
            } catch (HeroSmsException e) {
                if (CLOSED.contains(e.title)) {
                    throw e; // closed or expired: stop polling
                } else if (e.status == 429) {
                    sleepMs = (e.retryAfter > 0 ? e.retryAfter : 5) * 1_000; // documented for offers only
                } else if (e.status >= 500) {
                    lastError = e; // server error: retry until the deadline
                } else if (e.status == 400 || e.status == 404) {
                    lastError = null; // open activation, no SMS yet
                } else {
                    throw e; // any other error: stop polling
                }
            } catch (IOException e) {
                lastError = e; // timed-out or dropped request: retry until the deadline
            }
            long leftMs = Duration.ofNanos(deadline - System.nanoTime()).toMillis();
            if (leftMs <= 0) break;
            Thread.sleep(Math.min(sleepMs, leftMs));
            delayMs = Math.min(delayMs * 3 / 2, 10_000);
        }
        throw new IllegalStateException(
                "No SMS for activation " + activationId + " within " + timeout.toSeconds() + " s", lastError);
    }

    static String extractCode(JsonNode otp) {
        String code = text(otp.path("smsCode")).trim();
        if (!code.isEmpty()) return code;
        Matcher m = CODE.matcher(text(otp.path("smsText")));
        return m.find() ? m.group() : null;
    }

    private static String text(JsonNode node) {
        return node.isMissingNode() || node.isNull() ? "" : node.asText(); // NullNode.asText() is "null"
    }
}

GET /activations/{activationId}/otp/last returns smsCode (the code) and smsText (the whole message). Either one can be null. When smsCode is empty, the regex takes the first 4-8 digit number from smsText, so adjust it to your SMS template.
The API reference doesn't document the response before the first SMS. The poller therefore treats 400, 404 and an empty data as "not yet", unless title says the activation is closed. One side effect: a 404 caused by a wrong ID looks like a timeout.
A 5xx response, a timed-out request or a dropped connection is retried until the deadline. If polling runs out of time, the last such error becomes the exception's cause, so a proxy or DNS problem doesn't look like a missing SMS.
On 429 the loop honors Retry-After. The reference documents that header only for the offers endpoint, so the loop falls back to 5 seconds when it's absent.
The deadline uses System.nanoTime(), which doesn't jump when the runner's clock is adjusted.

Why not use Selenium's own FluentWait? It's generic, so new FluentWait<>(activationId) can poll any object, not just a driver. But Function.apply can't throw checked exceptions, and the polling interval is fixed. The 429 and closed-activation rules would end up wrapped and hidden. A plain loop is easier to read.

Step 3: the Selenium OTP test with JUnit 5 and explicit waits

// src/test/java/com/example/otp/OtpSignupTest.java
package com.example.otp;

import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.condition.EnabledIfEnvironmentVariable;
import org.openqa.selenium.By;
import org.openqa.selenium.StaleElementReferenceException;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

import java.time.Duration;
import java.util.List;

@Tag("real-sms")
@EnabledIfEnvironmentVariable(named = "HEROSMS_API_KEY", matches = ".+")
class OtpSignupTest {
    private static final By PHONE = By.cssSelector("[data-test=phone]");
    private static final By SEND_CODE = By.cssSelector("[data-test=send-code]");
    private static final By OTP = By.cssSelector("[data-test=otp]"); // the input, or a wrapper around one box per digit
    private static final By OTP_BOXES = By.cssSelector("[data-test=otp] input");
    private static final By VERIFY = By.cssSelector("[data-test=verify]");
    private static final By OTP_ERROR = By.cssSelector("[data-test=otp-error]");

    private WebDriver driver;
    private WebDriverWait wait;

    @BeforeEach
    void startBrowser() {
        ChromeOptions options = new ChromeOptions();
        options.addArguments("--headless=new", "--window-size=1366,900");
        driver = new ChromeDriver(options); // Selenium Manager finds or downloads the driver
        wait = new WebDriverWait(driver, Duration.ofSeconds(15)); // polls every 500 ms
        wait.ignoring(StaleElementReferenceException.class); // OTP boxes may re-render
    }

    @AfterEach
    void quitBrowser() {
        if (driver != null) driver.quit();
    }

    @Test
    void signsUpWithARealSmsCode() throws Exception {
        // Check the config before paying for a number
        String service = env("OTP_SERVICE", null);
        int country = Integer.parseInt(env("OTP_COUNTRY", null));
        String cap = env("HEROSMS_MAX_PRICE", "");
        Double maxPrice = cap.isEmpty() ? null : Double.valueOf(cap);
        String appUrl = env("APP_URL", null);

        HeroSmsClient sms = new HeroSmsClient(System.getenv("HEROSMS_API_KEY"));
        HeroSmsClient.Activation number = sms.buy(service, country, maxPrice);
        long boughtAt = System.nanoTime();
        boolean gotCode = false;
        try {
            driver.get(appUrl + "/signup");
            wait.until(ExpectedConditions.visibilityOfElementLocated(PHONE))
                .sendKeys("+" + number.phone());
            wait.until(ExpectedConditions.elementToBeClickable(SEND_CODE)).click();

            String code = SmsCodePoller.waitForCode(sms, number.id(), Duration.ofSeconds(120));
            gotCode = true;

            typeOtp(wrongCode(code)); // a wrong code must be rejected first
            wait.until(ExpectedConditions.elementToBeClickable(VERIFY)).click();
            wait.until(ExpectedConditions.visibilityOfElementLocated(OTP_ERROR));

            typeOtp(code);
            wait.until(ExpectedConditions.elementToBeClickable(VERIFY)).click();
            wait.until(ExpectedConditions.urlContains("/welcome")); // TimeoutException = test fails
        } finally {
            ActivationCleanup.release(sms, number.id(), gotCode, boughtAt);
        }
    }

    /** One input, or one box per digit: types by index, so it doesn't rely on auto-focus. */
    private void typeOtp(String code) {
        WebElement otp = wait.until(ExpectedConditions.visibilityOfElementLocated(OTP));
        if ("input".equalsIgnoreCase(otp.getTagName())) {
            otp.clear();
            otp.sendKeys(code);
            return;
        }
        wait.until(ExpectedConditions.numberOfElementsToBe(OTP_BOXES, code.length()));
        for (int i = 0; i < code.length(); i++) {
            int index = i;
            String digit = code.substring(i, i + 1);
            wait.until(d -> {
                List<WebElement> boxes = d.findElements(OTP_BOXES); // fresh lookup on every try
                if (boxes.size() <= index) return null; // mid re-render: keep waiting
                WebElement box = boxes.get(index);
                box.clear();
                box.sendKeys(digit);
                return true;
            });
        }
    }

    private static String wrongCode(String code) {
        return (code.startsWith("1") ? "2" : "1") + code.substring(1);
    }

    /** Blank counts as unset: GitHub passes an empty string for a missing variable. */
    private static String env(String name, String fallback) {
        String value = System.getenv(name);
        if (value != null && !value.isBlank()) return value.trim();
        if (fallback != null) return fallback;
        throw new IllegalStateException(name + " is not set");
    }
}

To run it locally, export the variables above and run mvn test -Dgroups=real-sms. If Surefire reports Tests run: 1, ... Skipped: 1, the key wasn't exported in that shell. Once the key is exported, a plain mvn test runs this test too and buys a number. For everyday runs, use mvn test -DexcludedGroups=real-sms.

@Tag("real-sms") lets Maven select or exclude the test. @EnabledIfEnvironmentVariable disables it when the key isn't set, so a PR build without the secret never buys a number.
The test reads all of its config before buy, and APP_URL has no fallback. A missing or malformed variable therefore fails the test before it pays for a number.
Every interaction goes through an explicit wait. WebDriverWait takes a Duration and polls every 500 ms. It ignores NotFoundException by default and throws TimeoutException when time runs out.
Use visibilityOfElementLocated for fields, elementToBeClickable for buttons, and numberOfElementsToBe to wait until every OTP box exists.
Don't set an implicit wait as well. The Selenium waits page warns against mixing implicit and explicit waits, because the resulting timeouts are unpredictable.
Split OTP inputs get one digit per box, by index, with a fresh lookup each time. If a re-render briefly leaves fewer boxes, the lambda returns null and the wait polls again. The wait also ignores StaleElementReferenceException, so a box that re-renders mid-typing is retried instead of failing the test.
The wrong-code check proves your app rejects bad input. If your app burns the code after one wrong attempt, drop those three lines. If it submits on the last digit, drop the Verify clicks.
For a login or SMS 2FA test, open your login page instead and type the code at the second-factor prompt. If your form has a separate country picker, strip countryPhoneCode (also in the purchase response) from the start of the number.
The Selenium Chrome docs use --headless=new. Chrome 132 removed the old headless mode, so plain --headless now does the same thing.

Step 4: finish or cancel the number in finally

// src/test/java/com/example/otp/ActivationCleanup.java
package com.example.otp;

import com.example.otp.HeroSmsClient.HeroSmsException;

import java.time.Duration;

final class ActivationCleanup {
    private ActivationCleanup() {}

    /** Finish if the code was used, cancel otherwise. Never throws: it must not hide the test failure. */
    static void release(HeroSmsClient sms, String id, boolean gotCode, long boughtAtNanos) {
        try {
            if (gotCode) {
                sms.finish(id); // the SMS arrived: close it, no refund
                return;
            }
            try {
                sms.cancel(id); // no SMS: cancel, the price goes back to your balance
            } catch (HeroSmsException e) {
                if ("OTP_RECEIVED".equals(e.title)) {
                    sms.finish(id); // the SMS came late
                } else if ("EARLY_CANCEL_DENIED".equals(e.title)) {
                    long minSec = e.info.path("minActivationTime").asLong(120);
                    long waitNanos = boughtAtNanos + Duration.ofSeconds(minSec + 5).toNanos() - System.nanoTime();
                    if (waitNanos > 0) Thread.sleep(Duration.ofNanos(waitNanos).toMillis());
                    try {
                        sms.cancel(id); // one more try, no more waiting
                    } catch (HeroSmsException again) {
                        if (!"OTP_RECEIVED".equals(again.title)) throw again;
                        sms.finish(id);
                    }
                } else {
                    throw e;
                }
            }
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            System.err.println("[otp] cleanup of " + id + " interrupted; check GET /activations");
        } catch (Exception e) {
            System.err.println("[otp] cleanup of " + id + " failed: " + e.getMessage());
        }
    }
}

The release call sits in the test's own finally, right after the purchase, so it runs whether the assertions pass or throw. @AfterEach only quits the browser.

If the code arrived, finish closes the activation, with no refund. If it didn't, DELETE cancels the activation and refunds the price.
The API refuses a cancel during a number's first minutes. It returns EARLY_CANCEL_DENIED with info.minActivationTime, which is about 2 minutes in the reference's examples. A test that fails within seconds therefore waits out that window plus 5 seconds, then tries once more.
If the SMS arrives late, the cancel fails with OTP_RECEIVED and the code calls finish instead. Cleanup branches on title, not on the HTTP status.
Cleanup errors are only logged. Throwing from finally would replace the real failure (say, a TimeoutException on the OTP field) with a less useful one.
If the runner itself dies, no Java code runs at all. Unused numbers are refunded to your balance, and GET /activations lists anything still open.

Step 5: run the Selenium Java test nightly in GitHub Actions

Real numbers cost money, so run this test on a schedule rather than on every push:

# .github/workflows/otp-nightly.yml
name: otp-nightly
on:
  schedule:
    - cron: '17 4 * * 1-5' # 04:17 UTC on weekdays; every run buys a number
  workflow_dispatch:

concurrency:
  group: otp-nightly
  cancel-in-progress: false # a canceled run may never reach the finally block

jobs:
  real-sms:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    steps:
      - name: Fail if the API key is missing
        env:
          HEROSMS_API_KEY: ${{ secrets.HEROSMS_API_KEY }}
        run: test -n "$HEROSMS_API_KEY" || { echo "::error::HEROSMS_API_KEY is not set"; exit 1; }
      - uses: actions/checkout@v7
      - uses: actions/setup-java@v6
        with:
          distribution: temurin
          java-version: '21'
          cache: maven
      - name: Real-SMS Selenium test
        env:
          HEROSMS_API_KEY: ${{ secrets.HEROSMS_API_KEY }}
          OTP_SERVICE: ${{ vars.OTP_SERVICE }}
          OTP_COUNTRY: ${{ vars.OTP_COUNTRY }}
          HEROSMS_MAX_PRICE: ${{ vars.HEROSMS_MAX_PRICE }}
          APP_URL: ${{ vars.APP_URL }}
        run: mvn -B test -Dgroups=real-sms
      - uses: actions/upload-artifact@v7
        if: failure()
        with:
          name: surefire-reports
          path: target/surefire-reports
          retention-days: 7

The first step fails the job if the secret is missing or misnamed. Without that step, GitHub passes an empty string, @EnabledIfEnvironmentVariable skips the test, and the nightly run goes green with Skipped: 1.
The key comes in through the step's env:, not the command line, as GitHub's secrets guide recommends. Surefire's forked test JVM inherits Maven's environment, so System.getenv sees the key.
Your PR workflow should run mvn -B test -DexcludedGroups=real-sms and not pass the key at all. GitHub doesn't pass secrets other than GITHUB_TOKEN to workflows triggered from forks, so a fork PR can't spend your balance. Don't move this job to pull_request_target, because that trigger does get secrets.
The ubuntu-latest image ships with Chrome and ChromeDriver, and Selenium Manager finds the driver itself, so the workflow needs no browser setup step.
Scheduled workflows have a few quirks:
- They run on the latest commit of the default branch.
- Times are in UTC unless you set a timezone.
- Under load they can start late or be dropped, especially at the top of the hour. That's why the cron uses minute 17.
- In a public repository, they are disabled after 60 days without activity.
Set HEROSMS_MAX_PRICE as a repository variable, so a price change can't quietly raise the cost of every run.
If you add failure screenshots, treat them as sensitive: they show the phone number and the code. Keep artifact retention short.

Troubleshooting

The code never arrives.
- Check that your app accepted the number with the +.
- Check that OTP_SERVICE matches the service that sends your SMS, since an activation is reserved for one service.
- If your app rejects certain prefixes, phoneException in the purchase body excludes them up front.
- POST /activations/{activationId}/replace gives you a free replacement number. If you use it, pass the activation it returns to ActivationCleanup.release, not the original ID.
TimeoutException on the OTP field. One of three things is usually wrong:
- The locator is wrong.
- The field is inside an iframe. Call driver.switchTo().frame(...) first.
- Your app shows the field only after the send-code request returns. Raise the wait for that one step instead of adding an implicit wait.
StaleElementReferenceException on split boxes. Your component re-renders after each digit. Re-query each box by index, as typeOtp does, and keep ignoring(StaleElementReferenceException.class) on the wait.
Skipped: 1 instead of a run. HEROSMS_API_KEY isn't set in that shell, so @EnabledIfEnvironmentVariable disabled the test. In CI, the guard step fails the job instead.
429 Too Many Requests. The poller honors Retry-After and backs off to 10 seconds. The concurrency group stops two scheduled runs from polling at the same time.
EARLY_CANCEL_DENIED in the log. This is expected when a test fails fast. Cleanup waits and retries once. If the retry is refused too, the number stays open until it expires.
Expired activation. Polling stops on ACTIVATION_NOT_ACTIVE, CANCELED, FINISHED or REFUNDED. A 2-minute poll sits well inside the ~20-minute activation. If you wait longer, compare against expiredAt.
SessionNotCreatedException about the ChromeDriver version. The usual cause is an old driver pinned with webdriver.chrome.driver, or a leftover WebDriverManager setup. Remove it and let Selenium Manager pick the matching driver.
Passes locally, fails headless. Headless Chrome may lay the page out differently. Set an explicit --window-size, as above, and take a screenshot on failure to see what the browser saw.

FAQ

Can I automate Selenium 2FA in Python instead? Yes, and the API calls are the same. My Python OTP post has the client and the polling loop with requests, so you can call them from a pytest Selenium test.

How do I automate email OTP in Selenium Java? It's the same loop with a different inbox. Create a test inbox with an email-testing API and poll it for the message with the same deadline loop. Then pull the code out with the same regex and type it with the same typeOtp. Only the client class changes.

Can I just skip OTP or CAPTCHA in tests? For most runs, yes. Selenium's docs recommend a test-only token, or 2FA disabled for test users. Keep one real-SMS run to check delivery. Never try to solve CAPTCHAs in automation; disable them in your test environment instead.

What about TOTP and Google Authenticator in Selenium? You don't need a phone for those. Store the shared secret as a CI secret and generate the current code at runtime with any RFC 6238 TOTP library.

How do I automate mobile OTP in Selenium Java? "Mobile OTP" usually means an SMS code sent to a phone number, which is exactly what this test handles. If the form lives in a native app rather than a web page, the same HeroSmsClient and poller work from an Appium test.

How much does a run cost? Each run pays for one number. The price depends on the service and country, and HEROSMS_MAX_PRICE caps it. A run that gets no SMS is canceled and refunded. If the cancel retry is refused too, the number stays open until it expires. A run that receives an SMS is paid even if your app then fails. A weekday schedule comes to about 22 numbers a month.

Use it on your own app only

Test only apps you own or are authorized to test. A rented number is yours only for the activation window, so don't tie it to an account you'll need later, and delete test users afterwards.

Wrap-up

Keep mocks and test tokens on every PR, and add one scheduled Selenium run against a real number. The PR runs prove the page works. The nightly run proves that a real user gets a code they can type into that page, in the country you launch in, with the SMS template you actually ship. If you'd rather have codes pushed to you than polled, the HeroSMS API reference documents an sms-incoming webhook. The Playwright post above shows that setup.

How do you handle OTP in Selenium today: a test token, a shared phone, or something else?

How to Automate OTP in Playwright: Real SMS Codes via Webhook in GitHub Actions

Willette — Mon, 26 Oct 2026 16:12:15 +0000

Disclosure: I work with HeroSMS, the virtual number provider used in the examples below. Written with AI tools; code and endpoints checked against the HeroSMS API reference.

If you've tried to automate OTP in Playwright, you've probably seen this in a nightly run:

✘ [chromium] › tests/signup.spec.ts:4:5 › sign-up with SMS code (30.0s)
    Test timeout of 30000ms exceeded.

The root cause: CI runners have no SIM. The usual fixes for a red Playwright OTP test each bite back:

A shared number with SMS forwarded by an old Android phone that reboots overnight.
Parallel workers hitting that same number and getting throttled.
A SKIP_OTP backend flag, the most dangerous fix: one bad config and it's live in production.

My previous post solved this in Python by polling a phone-number API. This is the Playwright OTP version in TypeScript: codes arrive by webhook, parallel workers never grab each other's codes, teardown refunds numbers that got no SMS, and it all runs in GitHub Actions.

When a Playwright OTP test needs a real number

Level	Good for	Risk	Cost
1. Mocks or the SMS gateway's sandbox	Every PR	Misses integration bugs	Mock upkeep
2. Code from staging DB or Redis	Backends you control	Runners often can't reach the DB	Test-only data layer
3. Real rented number + webhook	Nightly, pre-release, new countries	Carrier delays	A paid number per run

Fictional numbers belong to level 1: Firebase allows up to 10 "Phone numbers for testing", though Twilio says its test credentials don't work with Verify (see the previous post). Use all three levels, run level 3 on a schedule, and never ship an OTP-bypass flag: CI stays green while proving nothing about SMS.

How the Playwright OTP webhook flow works

Instead of polling, the provider calls you:

fixture  -> POST /activations (buy a number)
test     -> types it, clicks "Send code"; your app sends the SMS
HeroSMS  -> POST sms-incoming -> tunnel -> webhook-server.ts
server   -> writes .otp-inbox/<activationId>.json
fixture  -> waitForCode() resolves; the test submits the code
teardown -> finish (code used) or DELETE (no code, refund)

Three facts from the API reference drive the design:

Webhook URLs (HTTPS, up to 3) are set in the account profile, not per request, so every SMS on the account reaches you. If the main account is used for other work, give CI its own account and API key.
Your endpoint must answer 200 within 3 seconds, or the call is retried (about 7 times, 20–30 seconds apart).
Requests come from two fixed IPs: 84.32.223.53 and 185.138.88.87.

So: filter by activationId, always return 200, and run the receiver as its own process. Playwright workers share no memory, so the inbox lives on disk.

Install with npm i -D @playwright/test tsx @types/node (Node 20+).

A minimal HeroSMS client in TypeScript

Auth is an Authorization: ApiKey <token> header; errors are JSON with title, details and sometimes info.

// tests/herosms.ts
const API = 'https://hero-sms.com/api/v1';

export interface Activation {
  id: number;
  phone: number; // digits only, no "+"
  countryPhoneCode: number;
  expiredAt: string;
}
export type Offer = { counts: { total: number } };
type ErrorBody = { title?: string; details?: string; info?: Record<string, unknown> };

export class HeroSmsError extends Error {
  status: number;
  title: string;
  info?: Record<string, unknown>;
  constructor(status: number, body: ErrorBody) {
    super(`HTTP ${status} ${body.title ?? ''}: ${body.details ?? ''}`);
    this.status = status;
    this.title = body.title ?? '';
    this.info = body.info;
  }
}

export async function call<T>(method: string, path: string, body?: unknown): Promise<T | undefined> {
  const key = process.env.HEROSMS_API_KEY;
  if (!key) throw new Error('HEROSMS_API_KEY is not set');
  const res = await fetch(API + path, {
    method,
    headers: { Authorization: `ApiKey ${key}`, ...(body ? { 'Content-Type': 'application/json' } : {}) },
    body: body ? JSON.stringify(body) : undefined,
    signal: AbortSignal.timeout(15_000), // keep teardown from hanging
  });
  const text = await res.text();
  if (res.status >= 400) {
    let err: ErrorBody = {};
    try { err = JSON.parse(text) ?? {}; } catch { /* not JSON */ }
    throw new HeroSmsError(res.status, err);
  }
  return text ? (JSON.parse(text) as T) : undefined; // 204 -> undefined
}

export async function checkOffer(service: string, country: number): Promise<Offer | null> {
  const qs = new URLSearchParams({ services: service, countries: String(country) });
  try {
    const res = await call<{ data?: Record<string, Record<string, Offer>> }>('GET', `/activations/offers/sms?${qs}`);
    return res?.data?.[service]?.[String(country)] ?? null;
  } catch (e) {
    if (e instanceof HeroSmsError && e.status === 404) return null; // OFFER_NOT_FOUND
    throw e;
  }
}

export async function buyNumber(service: string, country: number, maxPrice?: number): Promise<Activation> {
  const res = await call<{ data: Activation[] }>('POST', '/activations', {
    service, country, amount: 1, verificationType: 'sms', ...(maxPrice ? { maxPrice } : {}),
  });
  if (!res?.data?.[0]) throw new Error('POST /activations returned no activation');
  return res.data[0];
}

export const finishActivation = (id: number) => call('POST', `/activations/${id}/finish`); // no refund
export const cancelActivation = (id: number) => call('DELETE', `/activations/${id}`); // refund

const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));

// Never throws: cleanup must not mask the test failure.
export async function release(a: Activation, gotOtp: boolean, boughtAt: number): Promise<void> {
  try {
    if (gotOtp) {
      await finishActivation(a.id);
      return;
    }
    try {
      await cancelActivation(a.id);
    } catch (e) {
      if (!(e instanceof HeroSmsError)) throw e;
      if (e.title === 'EARLY_CANCEL_DENIED') {
        const minSec = Number(e.info?.minActivationTime) || 120;
        await sleep(Math.max(0, boughtAt + minSec * 1000 + 5_000 - Date.now()));
        try {
          await cancelActivation(a.id);
        } catch (e2) {
          // The SMS may arrive during the wait: close the activation instead of leaving it open
          if (e2 instanceof HeroSmsError && e2.title === 'OTP_RECEIVED') await finishActivation(a.id);
          else throw e2;
        }
      } else if (e.title === 'OTP_RECEIVED') {
        await finishActivation(a.id); // late SMS
      } else {
        throw e;
      }
    }
  } catch (e) {
    const what = e instanceof HeroSmsError ? `HTTP ${e.status} ${e.title}` : (e as Error).name;
    console.warn(`cleanup ${a.id}: ${what}`);
  }
}

phone is an integer without +; the fixture prepends it. With a separate country picker, strip String(a.countryPhoneCode) from the front.
finish never refunds. DELETE refunds, but only "if the activation has not received any OTP". On an early-cancel refusal (EARLY_CANCEL_DENIED, minActivationTime: 120 seconds in the docs' example), release() waits and retries once; a late SMS (OTP_RECEIVED) gets finish. It branches on title, not HTTP status.
Plain fetch, not Playwright's request fixture, keeps the API key out of traces.
Use service ot ("Any other") and your launch country's ID (16 = United Kingdom). POST /activations/{id}/replace swaps a rejected number; phoneException excludes prefixes.

Webhook receiver: one inbox file per activationId

// tests/inbox.ts
import fs from 'node:fs';
import path from 'node:path';

// sms-incoming body (also has phoneFrom, service, country)
export interface SmsWebhookPayload {
  activationId: number;
  text: string | null;
  code?: string | null;
  receivedAt: string;
}

const INBOX_DIR = process.env.OTP_INBOX_DIR ?? path.resolve('.otp-inbox');

// sanitize: the id comes from outside
const inboxFile = (id: number | string) =>
  path.join(INBOX_DIR, `${String(id).replace(/[^\w-]/g, '_')}.json`);

function readSms(file: string): SmsWebhookPayload | null {
  try {
    return JSON.parse(fs.readFileSync(file, 'utf8')) as SmsWebhookPayload;
  } catch {
    return null; // not there yet
  }
}

export function saveSms(p: SmsWebhookPayload): void {
  fs.mkdirSync(INBOX_DIR, { recursive: true });
  const file = inboxFile(p.activationId);
  const prev = readSms(file);
  if (prev && prev.receivedAt > p.receivedAt) return; // retries repeat, resends are newer
  const tmp = `${file}.${process.pid}.tmp`;
  fs.writeFileSync(tmp, JSON.stringify(p));
  fs.renameSync(tmp, file); // atomic: a reader never sees half a file
}

export const extractCode = (p: SmsWebhookPayload) =>
  p.code || p.text?.match(/\b\d{4,8}\b/)?.[0];

export function waitForOtp(id: number, timeoutMs: number): Promise<string> {
  fs.mkdirSync(INBOX_DIR, { recursive: true });
  const file = inboxFile(id);
  return new Promise((resolve, reject) => {
    const watcher = fs.watch(INBOX_DIR, (_event, name) => {
      if (name === path.basename(file)) tryRead();
    });
    const timer = setTimeout(() => {
      watcher.close();
      reject(new Error(`No OTP for activation ${id} after ${timeoutMs} ms`));
    }, timeoutMs);
    function tryRead() {
      const sms = readSms(file);
      const code = sms ? extractCode(sms) : undefined;
      if (!code) return;
      watcher.close();
      clearTimeout(timer);
      resolve(code);
    }
    tryRead(); // the SMS may already be there
  });
}

// tests/webhook-server.ts
import http from 'node:http';
import { saveSms, type SmsWebhookPayload } from './inbox';

const PORT = Number(process.env.WEBHOOK_PORT ?? 3999); // pick a port your app doesn't use
const HOOK_PATH = process.env.WEBHOOK_PATH || '/hooks/herosms'; // use a long random path
const MAX_BODY = 64 * 1024;

http.createServer((req, res) => {
  if (req.method === 'GET' && req.url === '/health') {
    res.writeHead(200).end('ok');
    return;
  }
  if (req.method !== 'POST' || req.url !== HOOK_PATH) {
    res.writeHead(404).end();
    return;
  }
  let body = '';
  req.setEncoding('utf8'); // non-Latin SMS text can span chunks
  req.on('data', (chunk: string) => {
    body += chunk;
    if (body.length > MAX_BODY) req.destroy();
  });
  req.on('end', () => {
    try {
      const p = JSON.parse(body) as SmsWebhookPayload;
      if (Number.isFinite(p?.activationId)) saveSms(p);
    } catch (err) {
      // name only: the body holds SMS text
      console.warn('[webhook] bad payload:', err instanceof Error ? err.name : 'unknown');
    }
    res.writeHead(200, { 'Content-Type': 'application/json' }).end('{}'); // always 200, fast
  });
}).listen(PORT, '127.0.0.1');

Webhook field names differ from REST: code and text here, smsCode and smsText there. code can be null or missing, so extractCode falls back to a regex on text; adapt it to your template.
Duplicates and broken payloads still get a fast 200; a non-200 only buys retries. Never log text, and add .otp-inbox/ to .gitignore: each file holds a full SMS.
Enforce the IP allowlist at the tunnel edge, not in Node: behind a tunnel, remoteAddress is the proxy and X-Forwarded-For can be spoofed. The reference documents no webhook signature, so the allowlist plus an unguessable WEBHOOK_PATH are your controls.

The Playwright OTP fixture: buy, wait, always clean up

// tests/otp-fixture.ts
import { test as base, expect } from '@playwright/test';
import { buyNumber, checkOffer, release } from './herosms';
import { waitForOtp } from './inbox';

export type Otp = {
  phone: string;
  activationId: number;
  waitForCode: (timeoutMs?: number) => Promise<string>;
};

export const test = base.extend<{ otp: Otp }>({
  otp: async ({}, use) => {
    // validate config before spending money
    const service = process.env.OTP_SERVICE || 'ot';
    const country = Number.parseInt(process.env.OTP_COUNTRY ?? '', 10);
    const maxPrice = process.env.HEROSMS_MAX_PRICE ? Number(process.env.HEROSMS_MAX_PRICE) : undefined;
    if (!Number.isInteger(country) || Number.isNaN(maxPrice)) {
      throw new Error('OTP_COUNTRY must be a country ID, HEROSMS_MAX_PRICE a number');
    }
    const offer = await checkOffer(service, country); // free, nothing bought yet
    if (!offer || offer.counts.total === 0) throw new Error(`No ${service} numbers in country ${country}`);

    const a = await buyNumber(service, country, maxPrice);
    const boughtAt = Date.now();
    let gotOtp = false;
    try {
      await use({
        phone: `+${a.phone}`,
        activationId: a.id,
        waitForCode: async (ms = 120_000) => {
          const code = await waitForOtp(a.id, ms);
          gotOtp = true;
          return code;
        },
      });
    } finally {
      await release(a, gotOtp, boughtAt);
    }
  },
});

export { expect };

Code after use() runs even when the test fails or times out (fixtures docs).
Fixture teardown gets its own timeout, equal to the test timeout, so the early-cancel wait (up to about two minutes) fits.

Writing the Playwright OTP test

// tests/signup.spec.ts
import { test, expect } from './otp-fixture';

test.skip(!process.env.HEROSMS_API_KEY, 'real-SMS test: set HEROSMS_API_KEY');

test('sign-up with a real SMS OTP', { tag: '@real-sms' }, async ({ page, otp }) => {
  await page.goto('/signup');
  await page.getByLabel('Phone number').fill(otp.phone);
  await page.getByRole('button', { name: 'Send code' }).click();

  const code = await otp.waitForCode(); // 120 s by default
  await page.getByLabel('Verification code').fill(code);
  await page.getByRole('button', { name: 'Verify' }).click();
  await expect(page).toHaveURL(/\/welcome/);
});

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: 'tests',
  timeout: 240_000, // the OTP wait alone is 120 s
  reporter: [['list'], ['html', { open: 'never' }]],
  use: {
    baseURL: process.env.APP_URL, // your app, never the webhook receiver
    trace: 'retain-on-failure',
  },
  webServer: {
    command: 'npx tsx tests/webhook-server.ts',
    url: `http://127.0.0.1:${process.env.WEBHOOK_PORT ?? 3999}/health`,
    reuseExistingServer: !process.env.CI,
  },
});

Without the secret, the file-level test.skip keeps PR runs from buying anything; the workflow selects it with --grep @real-sms.
webServer waits until url returns 2xx, 3xx or 400–403, hence /health. The docs prefer url over the older port; set use.baseURL explicitly.
International SMS can be slow: 120 s for the code, 240 s for the test; keep both well under the activation's expiredAt.
--workers=N is safe as long as counts.total covers N: each worker waits on its own activationId.

Running Playwright OTP tests in GitHub Actions

Runners don't accept inbound connections, and the webhook URL is fixed in your account. So reserve an ngrok domain, register https://<domain><WEBHOOK_PATH> once in the HeroSMS profile, and open the tunnel in each job with --url. A traffic policy admits only the two webhook IPs:

# .github/ngrok-policy.yml
on_http_request:
  - actions:
      - type: restrict-ips
        config:
          enforce: true
          allow:
            - 84.32.223.53/32
            - 185.138.88.87/32

# .github/workflows/e2e-otp.yml
name: e2e-otp
on:
  workflow_dispatch:
  schedule:
    - cron: '0 2 * * 1-5' # every run buys real numbers

concurrency:
  group: e2e-otp
  cancel-in-progress: false

jobs:
  e2e:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    env:
      WEBHOOK_PORT: 3999
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: 24
          cache: npm
      - run: npm ci
      - run: npx playwright install --with-deps chromium

      - name: Install ngrok
        run: |
          curl -sSL https://ngrok-agent.s3.amazonaws.com/ngrok.asc | sudo tee /etc/apt/trusted.gpg.d/ngrok.asc >/dev/null
          echo "deb https://ngrok-agent.s3.amazonaws.com bookworm main" | sudo tee /etc/apt/sources.list.d/ngrok.list
          sudo apt-get update && sudo apt-get install -y ngrok

      - name: Start tunnel
        env:
          NGROK_AUTHTOKEN: ${{ secrets.NGROK_AUTHTOKEN }}
          WEBHOOK_URL: ${{ vars.WEBHOOK_URL }} # https://<reserved-domain>
        run: |
          ngrok http "127.0.0.1:$WEBHOOK_PORT" --url "$WEBHOOK_URL" \
            --traffic-policy-file .github/ngrok-policy.yml > "$RUNNER_TEMP/ngrok.log" 2>&1 &
          for i in $(seq 1 30); do
            # policy blocks the runner's IP: 403 = tunnel online
            [ "$(curl -s -o /dev/null -w '%{http_code}' "$WEBHOOK_URL/health")" = 403 ] && exit 0
            sleep 1
          done
          cat "$RUNNER_TEMP/ngrok.log"; exit 1

      - name: Real-SMS tests
        env:
          HEROSMS_API_KEY: ${{ secrets.HEROSMS_API_KEY }}
          WEBHOOK_PATH: ${{ secrets.WEBHOOK_PATH }}
          HEROSMS_MAX_PRICE: ${{ vars.HEROSMS_MAX_PRICE }}
          OTP_SERVICE: ${{ vars.OTP_SERVICE }}
          OTP_COUNTRY: ${{ vars.OTP_COUNTRY }}
          APP_URL: ${{ vars.APP_URL }}
          OTP_INBOX_DIR: ${{ runner.temp }}/otp-inbox
        run: npx playwright test --grep @real-sms --workers=2

      - uses: actions/upload-artifact@v7
        if: failure()
        with:
          name: playwright-report
          path: playwright-report
          retention-days: 1 # holds phone numbers and OTPs

Keys and tokens go in secrets, other settings in vars. OTP_INBOX_DIR is set on the step because job-level env can't read the runner context.
concurrency matters: one domain means one tunnel, and you get only 3 webhook URLs.
See the Playwright CI guide for the rest.
Local dry run: ngrok http 127.0.0.1:3999 --traffic-policy-file .github/ngrok-policy.yml with a random WEBHOOK_PATH, register the temporary URL plus path, run npx playwright test, then remove it.
ngrok shows POST 200 but the test times out? Check the activationId/file-name match, not the network.

Can't open a tunnel (say, a locked-down self-hosted runner)? Poll GET /activations/{id}/otp/last every 5 seconds, treating 400, 404 and empty data as "not yet", as the Python post does.

Responsible use and secrets hygiene

Test only apps and accounts you own or are authorized to test. Using rented numbers to get around other services' sign-up limits, mass-create accounts or dodge verification typically violates their terms.
A rented number is yours only for the activation window: don't tie it to anything you'd need to recover, and delete test users afterwards.
Treat OTPs as secrets. fill() values end up in traces and HTML reports, so upload reports only on failure, with 1-day retention. Don't print codes; if you must, emit ::add-mask::<code> first (logs only, not artifacts).
API key in CI secrets and plain fetch only; set HEROSMS_MAX_PRICE. And no OTP-bypass flags, not even "staging only".

Wrap-up

Mocks cover every PR; one scheduled real-number Playwright OTP run catches what they can't: delivery failures, broken SMS templates and per-country quirks. The examples run against HeroSMS; the webhook schema and error titles are in the API reference.

How do you handle OTP in Playwright today: mocks, a test number, or a phone in a drawer?

How to Test an SMS OTP Flow End-to-End in Python with a Real Phone Number API

Willette — Sat, 17 Oct 2026 13:31:32 +0000

Disclosure: I work with HeroSMS, the virtual number provider used in the examples below. Written with AI tools; code and endpoints checked against the HeroSMS API reference.

Your sign-up form sends a one-time code by SMS. The unit tests mock the SMS client, so they pass. Then a tester abroad reports that the code never arrived, or arrived after the form timed out. CI couldn't catch that, because it never sent a real message.

Here's how to test that flow end-to-end in Python. The script rents a real number through an SMS verification API, lets your app send the code, reads it by polling or webhook, submits it back, and cleans up so failed runs get refunded.

When to use real numbers vs. test numbers and mocks

Use the cheapest tool that answers your question:

Mocks of your SMS client. For unit and most integration tests. Fast and deterministic, but they prove your logic, not delivery.
Twilio test credentials. Twilio won't charge you, reach real phones or fire status callbacks, and the magic From number +15005550006 passes validation. Useful for checking your request code, though Twilio says test credentials don't work with Verify.
Firebase Auth test phone numbers. Under "Phone numbers for testing" in the Phone sign-in provider settings, you can add up to 10 fictional numbers, each with a fixed 6-digit code. No SMS is sent, no quota is used, and repeated runs aren't throttled. Good for UI tests.
Real numbers. These answer "does a real SMS arrive, and does the whole flow accept it?" Use them for a pre-release staging smoke test, after changing SMS providers or templates, or when launching in a new country.

Real-number runs cost money and take minutes, so put them on a schedule, not on every commit.

Setup

You buy an activation, one number reserved for one service in one country. You need a HeroSMS API key and requests:

pip install requests
export HEROSMS_API_KEY="your-key"
export HEROSMS_COUNTRY=16    # country ID; 16 is the United Kingdom
export APP_SIGNUP_URL="<your staging sign-up endpoint>"
export APP_VERIFY_URL="<your staging verify endpoint>"
# optional: HEROSMS_SERVICE (default "ot"), HEROSMS_MAX_PRICE, OTP_TIMEOUT

The REST API lives at https://hero-sms.com/api/v1 and expects an Authorization: ApiKey <token> header. Errors carry title and details. The flow uses five endpoints:

Step	Endpoint
Check price and stock	`GET /activations/offers/sms?services=ot&countries=16`
Buy a number	`POST /activations`
Read the OTP	`GET /activations/{activationId}/otp/last`
Code received, close it	`POST /activations/{activationId}/finish`
No code, cancel and refund	`DELETE /activations/{activationId}`

For your own app, use the service code ot ("Any other") and the country you're launching in. The API docs list all codes and country IDs.

The Python script: receive the OTP over the API

# otp_e2e.py - pip install requests
import os
import re
import sys
import time

import requests

API = "https://hero-sms.com/api/v1"
session = requests.Session()
session.headers["Authorization"] = f"ApiKey {os.environ['HEROSMS_API_KEY']}"
CLOSED = {"ACTIVATION_NOT_ACTIVE", "CANCELED", "FINISHED", "REFUNDED"}


class HeroSMSError(RuntimeError):
    def __init__(self, resp):
        try:
            body = resp.json()
        except ValueError:
            body = {}
        if not isinstance(body, dict):
            body = {}
        try:
            self.retry_after = int(resp.headers.get("Retry-After") or 0)
        except ValueError:  # an HTTP-date instead of seconds
            self.retry_after = 0
        self.status = resp.status_code
        self.title = body.get("title", "")
        details = body.get("details", resp.text[:200])
        super().__init__(f"HTTP {self.status} {self.title}: {details}")


def call(method, path, **kwargs):
    r = session.request(method, API + path, timeout=15, **kwargs)
    if r.status_code >= 400:
        raise HeroSMSError(r)
    return r.json() if r.content else None


def check_offer(service, country):
    try:
        resp = call("GET", "/activations/offers/sms",
                    params={"services": service, "countries": country})
    except HeroSMSError as e:
        if e.status != 404:  # 404 = OFFER_NOT_FOUND
            raise
        resp = None
    data = (resp or {}).get("data") or {}
    offer = (data.get(service) or {}).get(str(country))
    if not offer or offer["counts"]["total"] == 0:
        sys.exit(f"No {service} numbers in country {country} right now")
    return offer


def buy_number(service, country, max_price=None):
    body = {"service": service, "country": country,
            "amount": 1, "verificationType": "sms"}
    if max_price is not None:
        body["maxPrice"] = max_price
    return call("POST", "/activations", json=body)["data"][0]


def wait_for_otp(activation_id, timeout=300, interval=5):
    deadline = time.monotonic() + timeout
    while time.monotonic() < deadline:
        try:
            resp = call("GET", f"/activations/{activation_id}/otp/last")
            otp = (resp or {}).get("data") or {}
            if otp.get("smsCode") or otp.get("smsText"):
                return otp
        except HeroSMSError as e:
            if e.status == 429:
                time.sleep(e.retry_after or interval)
                continue
            # 400/404 = no SMS yet, unless the activation is closed
            if e.status not in (400, 404) or e.title in CLOSED:
                raise
        time.sleep(interval)
    return None


def release(activation_id, got_sms):
    try:
        if got_sms:
            call("POST", f"/activations/{activation_id}/finish")
        else:
            call("DELETE", f"/activations/{activation_id}")  # cancel + refund
    except (HeroSMSError, requests.RequestException) as e:
        print(f"Cleanup for {activation_id} failed: {e}", file=sys.stderr)


# Swap in your app's own calls (staging API or Playwright).
def trigger_signup(phone):
    r = requests.post(os.environ["APP_SIGNUP_URL"],
                      json={"phone": phone}, timeout=15)
    r.raise_for_status()


def verify_code(phone, code):
    r = requests.post(os.environ["APP_VERIFY_URL"],
                      json={"phone": phone, "code": code}, timeout=15)
    r.raise_for_status()  # add asserts on the response body


def main():
    service = os.getenv("HEROSMS_SERVICE", "ot")  # "ot" = "Any other"
    country = int(os.environ["HEROSMS_COUNTRY"])  # 16 = United Kingdom
    max_price = os.getenv("HEROSMS_MAX_PRICE")

    offer = check_offer(service, country)
    print(f"{offer['counts']['total']} numbers, "
          f"price {offer['prices']['default']}")
    activation = buy_number(service, country,
                            float(max_price) if max_price else None)
    act_id = activation["id"]

    otp = None
    try:
        phone = f"+{activation['phone']}"
        print(f"Activation {act_id}: {phone}, "
              f"expires {activation.get('expiredAt')}")
        trigger_signup(phone)
        timeout = int(os.getenv("OTP_TIMEOUT", "300"))
        otp = wait_for_otp(act_id, timeout=timeout)
        if otp is None:
            raise TimeoutError(f"No SMS for {phone} within {timeout}s")
        match = re.search(r"\b\d{4,8}\b", otp.get("smsText") or "")
        code = otp.get("smsCode") or (match.group() if match else None)
        if not code:
            raise AssertionError("SMS has no code; check your template")
        print(f"SMS from {otp.get('phoneFrom')}, code {code[:1]}***")
        verify_code(phone, code)
    finally:
        release(act_id, got_sms=otp is not None)


if __name__ == "__main__":
    main()

What's going on:

POST /activations returns a data array (amount goes up to 10 for parallel tests). phone comes without the +; strip the countryPhoneCode prefix if your form has a separate country picker. A standard activation lasts 20 minutes (see expiredAt), so keep OTP_TIMEOUT well under that.
otp/last returns smsCode (the parsed code) and smsText (the full message); either can be null. The reference doesn't document the response before the first SMS, so the script treats 400, 404 and an empty data as "not yet". It backs off on 429 and stops on a closed-activation title like ACTIVATION_NOT_ACTIVE or any other error. A 404 from a wrong ID will look like a timeout. If smsCode is missing, a regex reads the text; an SMS with no code fails the test instead of timing out.
verify_code submits the code, which turns a delivery check into a flow test. Assert whatever "signed up" means in your app.
Cleanup runs in finally. With no SMS, DELETE cancels and refunds. After an OTP, a regular activation can't be cancelled, so the script calls finish (not refunded). The docs say a number can't be cancelled in its first 2 minutes (EARLY_CANCEL_DENIED), so a sign-up call that fails within seconds can't cancel right away. release() logs that, plus network errors, without hiding the original failure; an activation that never gets an SMS is refunded automatically.
trigger_signup and verify_code use plain requests, not the API session, so your HeroSMS key never reaches your app. If your app rejects a number, POST /activations/{activationId}/replace swaps it, and phoneException in the purchase body excludes prefixes up front.

Receive the OTP by webhook instead of polling

Polling is fine for one nightly test. For many parallel runs, have the provider push messages instead. The sms-incoming webhook POSTs JSON to up to three HTTPS URLs set in your account profile.

# webhook.py - pip install flask
from flask import Flask, request

app = Flask(__name__)
HEROSMS_IPS = {"84.32.223.53", "185.138.88.87"}
latest = {}  # activationId -> newest SMS (use Redis or a DB for real)


@app.post("/hooks/herosms/sms")
def sms_incoming():
    # Called over HTTPS only: run behind your TLS proxy and take the
    # client IP from the proxy's header instead of remote_addr.
    if request.remote_addr not in HEROSMS_IPS:
        return "", 403
    event = request.get_json(silent=True) or {}
    act_id = event.get("activationId")
    if act_id is not None:
        prev = latest.get(act_id) or {}
        # Retries repeat an SMS; resends are newer. Keep the newest.
        if (event.get("receivedAt") or "") >= (prev.get("receivedAt") or ""):
            latest[act_id] = event
    return "", 200  # within 3 seconds

Things to get right:

Reply 200 within 3 seconds. Anything else counts as a failure and is retried at least 7 times, 20–30 seconds apart, so keep the handler idempotent and return 200 for duplicates too.
The payload has id, activationId, phoneFrom, service, text, code, country and receivedAt. The names differ from REST, which uses smsCode and smsText.
The docs publish these source IPs but no request signature, so the allowlist is your check. Your test then reads your own store; cleanup stays the same. Keep polling as a fallback, since a staging box behind a VPN often can't receive webhooks.

Use it on your own apps only

Test only apps you own or are authorized to test. Using rented numbers to get around other services' sign-up limits, mass-create accounts or dodge verification rules typically violates their terms.
A rented number is yours only for the activation window. Never tie it to an account you'll need to recover, and delete the test accounts you created with it.
Treat OTPs in logs like secrets. Anyone with CI log access can read a live code while it's still valid, which is why the script masks it.
Keep the API key in a CI secret, and set HEROSMS_MAX_PRICE to cap the price per number.

Wrap-up

Keep mocks and test numbers for everyday CI, and run one real-number check on a schedule to catch delivery problems.

The examples run against HeroSMS. The full reference, including the webhook schema, is in the API docs.