Building a Fleet of 16 Unattended Bank & Payment-Portal Automation Bots

Aman upadhyay — Mon, 31 Aug 2026 06:25:16 +0000

How I replaced manual daily logins across 16 financial portals with a reliable, observable automation platform — and what it taught me about OTP relays, CAPTCHA-solving, and designing for failure, for my own use cases.

The Problem

Every day, the operations team had to log into a growing list of bank and payment-aggregator portals, manually pull transaction and statement reports, and email them out. It was repetitive, error-prone, and didn't scale — every new institution meant another portal, another login flow, another CAPTCHA, another OTP.

I set out to eliminate that manual work entirely, end to end: login → download → validate → deliver, with no human in the loop during steady state.

What started as six integrations grew into a fleet of 16 independent automation bots, each fully automating a different portal, plus three reusable platform capabilities I had to build from scratch because nothing off-the-shelf solved them: an OTP Relay service, a CAPTCHA automation toolkit, and a Unified Reporting Dashboard.

Architecture, at a Glance

Every bot in the fleet follows the same underlying pattern:

orchestrator → portal client → verifier → email client → idempotent state store
Browser automation: Playwright driving headless Chromium (mix of async/sync APIs depending on the integration)
Data processing: pandas + openpyxl/xlrd, with multi-format parsers since "Download as XLS" buttons on these portals often produce something that isn't really an XLS
Delivery: Gmail API (OAuth2) for most integrations; Playwright-driven Gmail web-UI automation with a saved session for a few
Scheduling: predominantly cron — either "every 30 minutes within business hours" or "one daily run with an in-process retry window"
Web/API layer: FastAPI + uvicorn, exposing trigger and status endpoints that a central dashboard consumes

This common shape is what let me go from six integrations to sixteen without the engineering cost scaling linearly — onboarding a new portal is now a matter of days, not weeks.

Innovation #1: The OTP Relay

OTP-gated logins were the single hardest blocker to unattended automation. My solution now runs as five independent lanes:

An Android app sits on a phone holding the registered SIM and filters incoming SMS against known sender IDs.
Matching messages are forwarded over a secure tunnel to a lightweight FastAPI receiver.
The receiver extracts the OTP via regex and writes it to a local spool.
The automation polls the spool for a fresh, unconsumed OTP at the exact moment the portal asks for one, submits it, and marks it consumed.

Freshness is computed against login start time, so a stale/leftover code is never reused.

One hard rule shaped the entire design: the relay never logs or stores the OTP or the raw SMS body. Receivers bind to loopback only — the tunnel is the only path to the outside world, never an exposed port.

python

simplified sketch of the polling contract

def get_fresh_otp(login_started_at: float, timeout_s: int = 60) -> str | None:
deadline = time.monotonic() + timeout_s
while time.monotonic() < deadline:
envelope = spool.peek_unconsumed()
if envelope and envelope.received_at >= login_started_at:
spool.mark_consumed(envelope.id)
return envelope.otp
time.sleep(1)
return None

Innovation #2: A CAPTCHA Toolkit, Not a Single Solver

I started with one OCR solver for one portal. It's now a toolkit of three distinct strategies, chosen per portal, because a wrong CAPTCHA doesn't just fail silently — it costs a login submission, and submissions are what lock corporate accounts.

Strategy How it works Where it fits
DOM-text read (deterministic) The "CAPTCHA" is actually rendered as DOM text — read the label, strip whitespace Always preferred where the portal allows it — turns login from probabilistic to deterministic
Image OCR Screenshot the CAPTCHA element, OCR with an alphanumeric allowlist Bounded retries per run, backed by the outer cron cadence
Audio CAPTCHA via Whisper Portal's own accessibility audio track, transcribed with Whisper Escalation path when image OCR structurally cannot recover the answer

That last one deserves its own explanation, because it's the most interesting failure mode I found.

When OCR Can't Fix Itself

On one portal, my best OCR ensemble scored 7/9 exact matches — but 8/9 if you ignored letter case. The accuracy ceiling was letter case, and it wasn't recoverable from pixels: the CAPTCHA randomizes each glyph's size and baseline independently, so relative height gives no signal (I measured an uppercase letter shorter than a lowercase one from another sample). Worse, the OCR engine reported full confidence on readings whose case was simply wrong — so confidence couldn't flag the error either.

The fix wasn't a better model. It was a different channel: the portal's own accessibility audio CAPTCHA announces case explicitly ("Lower case b. Upper case B. Six."). So the design became an escalation, not a retry — image OCR gets exactly one attempt; every retry after that goes to audio.

I also learned, the hard way, that a fallback wired to fire "only when the primary fails loudly" will never fire — an OCR model always returns something. The trigger has to be the rejected login attempt itself.

Choosing an Engine With Data, Not Instinct

Before committing to an OCR approach, I benchmarked options against 10 real CAPTCHAs pulled from a live portal:

Engine Exact match rate
Two-model ensemble (custom preprocessing, majority vote) 7/9
Single raw pass 5/9
General OCR engine A 3/9
General OCR engine B 1/9

General-purpose document/scene-text OCR is simply not competitive against adversarial verification-code imagery — worth knowing before reaching for one by default.

Innovation #3: A Read-Only Observability Dashboard

With 16 independently scheduled bots, the real question stopped being "can each bot work?" and became "do I actually know what happened today, across all of them?"

I built a unified dashboard — FastAPI backend, React/TypeScript frontend — governed by one rule: adapters read source state or read-only APIs; they never execute pipeline commands. Onboarding a bot to the dashboard means writing an adapter for that bot's state, not exposing new control endpoints on the bot.

Two normalization rules turned out to be the difference between a dashboard people trust and one that cries wolf:

A day with no transactions is NO_ACTIVITY, not FAILED. Several bots deliberately send nothing on a quiet day, and treating that as a failure trains people to ignore real alerts.
Dry runs are skipped, not counted as successes — a dry run proves the pipeline works but delivers nothing, and counting it paints a green day that never actually happened.

Designing for Failure (Because Portals Fail Constantly)

A few reliability invariants now hold across the entire fleet, earned the hard way:

Cron cadence is the retry loop. Nothing waits in-process for hours; all cross-run state lives on disk.
A zero exit code does not mean a report was sent. Idempotent skips, retries, and successes all exit cleanly — they mean different things.
Per-account failure isolation. One bad account never blocks the rest of a run.
Zero successes ⇒ do not email, and do not mark the day complete. I found this one the hard way: a run with zero successes emailed an empty report and closed the day — permanently suppressing that day's real data. Every integration since has been designed against exactly that failure mode.
Never retry a rejected credential. Several portals lock an account after a handful of wrong attempts, so live testing leans on read-only probes and dry runs rather than burning real logins.
Empty-but-valid is a business outcome, not a failure. "No transactions today" must never be retried or classified as "portal down."

One portal reliably killed its session after only one or two accounts. Rather than fight it, I redesigned around it: the run is sharded across multiple login sessions, driven by a durable per-account ledger. A session death ends only that shard; the next shard picks up stranded accounts first; the day's email is composed from the ledger — the single source of truth across every shard — and sent exactly once.

Where I Am Now

16 live/built integrations, up from 6
55+ accounts under active automation, across fixed rosters and portals that discover their account lists dynamically at runtime
21 distinct report types automated end-to-end
3 reusable platform capabilities — OTP Relay (5 lanes), a 3-strategy CAPTCHA toolkit with a measured engine comparison, and a unified observability dashboard
One common architecture pattern, proven across sixteen independent codebases, that turned onboarding a new portal from a multi-week effort into a multi-day one

The manual work — logging into sixteen separate portals every day, by hand, to download and email reports — is gone.

What's Next

I'm now working on retrofitting my most mature integration's engineering practices — centralized selectors, architectural decision records, CI gates, written deployment runbooks — across the rest of the fleet, and moving the dashboard from a monitoring tool into the primary way this data gets consumed, with email becoming the fallback rather than the interface.

If you're building something similar — OTP automation, CAPTCHA-solving pipelines, or fleets of unattended browser bots — I'd love to compare notes in the comments.

DEV Community: Aman upadhyay

Building a Fleet of 16 Unattended Bank & Payment-Portal Automation Bots

simplified sketch of the polling contract