How to Check That Your OTP SMS and Emails Actually Arrive (Synthetic Monitoring in CI)

#testing #devops #monitoring #api

Your uptime monitor says the login API returned 200. Your SMS provider's dashboard says "sent". And yet users can't log in, because the verification code never showed up.

That gap is common and hard to see: an expired sender ID, a template that started landing in spam, a carrier quietly filtering a route, a provider incident where the API keeps returning success. Delivery receipts help, but they come from the same system you're trying to verify.

The reliable way to know is a synthetic check: on a schedule, trigger a real OTP send to a real phone number or inbox you control, and measure whether (and how fast) it arrives.

This post shows how to run that check from CI or cron. Disclosure: I built otp-watch, the small API used below, because I wanted this for my own projects and didn't find a simple version.

How the check works

Ask for a target: a fresh email inbox or a real UK mobile number.
Trigger your own app's OTP send to that target (the same code path real users hit).
Poll until the result is received (with latency in ms) or timed_out.
Fail the job on timed_out, so your existing CI alerting does the paging.

1. Get an API key

curl -s -X POST https://otpwatch.flo-voice1.com/keys \
  -H 'content-type: application/json' \
  -d '{"email":"you@example.com"}'
# => {"key":"..."}

2. Start a check

curl -s -X POST https://otpwatch.flo-voice1.com/checks \
  -H "authorization: Bearer $OTP_WATCH_KEY" \
  -H 'content-type: application/json' \
  -d '{"channel":"email","timeoutSeconds":120}'

The response gives you a check id and the target to send to:

{
  "id": "GEUOHivIN-F_i4AN",
  "channel": "email",
  "status": "pending",
  "target": "0a625bab1a59@receivemail.dev",
  "timeoutSeconds": 120
}

Use "channel":"sms" and target is a real UK mobile number on a physical SIM (not VoIP), so it receives codes the way your users' phones do.

3. Trigger your send, then poll

curl -s https://otpwatch.flo-voice1.com/checks/$ID \
  -H "authorization: Bearer $OTP_WATCH_KEY"
# => {"status":"received","latencyMs":4210,...}
# or {"status":"timed_out",...}

timeoutSeconds can be anything from 10 to 600 (default 120).

A complete CI step

This is the whole thing as a shell step you can drop into GitHub Actions, GitLab CI, or a cron job. Replace the middle line with however your app triggers a verification send (a test endpoint, a signup call, a Playwright run).

set -euo pipefail
API=https://otpwatch.flo-voice1.com

CHECK=$(curl -s -X POST $API/checks \
  -H "authorization: Bearer $OTP_WATCH_KEY" \
  -H 'content-type: application/json' \
  -d '{"channel":"email","timeoutSeconds":120}')
ID=$(echo "$CHECK" | jq -r .id)
TARGET=$(echo "$CHECK" | jq -r .target)

# Trigger YOUR app's OTP send to $TARGET here, e.g.:
curl -s -X POST https://your-app.example/api/test/send-otp -d "{\"to\":\"$TARGET\"}"

until S=$(curl -s $API/checks/$ID -H "authorization: Bearer $OTP_WATCH_KEY" | jq -r .status); [ "$S" != pending ]; do
  sleep 5
done

echo "OTP delivery: $S"
[ "$S" = received ] || exit 1

Run it every 15 or 30 minutes and you'll know about a broken OTP path before your users tell you.

Cost

Email checks are free and unlimited (rate-limited to 30 checks per hour per key).
Each key gets 3 free one-off SMS checks; GET /usage shows what's left.
For frequent SMS checks there's a monitor: one dedicated number for 30 days, unlimited checks against it, €29/month. Reusing one number is also what makes scheduled SMS checks practical, since you're not renting a new number every run.

What this doesn't replace

A synthetic check tells you the path works for one test number or inbox. It doesn't replace your provider's delivery receipts or per-country monitoring at scale. It's the cheap, independent "does a real code actually arrive" signal that's missing from most setups.

Links

Questions or edge cases from your own OTP setup are very welcome in the comments.

DEV Community