How I redesigned a distributed architecture to survive silent drops, strict rate limits, and SMTP chaos using just one IP.
When I first architected an email verification system, the mental model was deceptively simple: validate the syntax, check the DNS records, ping the SMTP server, and return the result.
Client → API → Direct SMTP Call. Done, right?
Wrong.
As soon as I ran this naive architecture under actual load, the system didn't just slow down—it shattered. SMTP servers began silently dropping connections. Domains started returning ambiguous "unknown" statuses. My CPU spiked, yet throughput flatlined.
My immediate instinct was the classic developer fallacy: Just add more workers. It made things worse. Adding more workers to a bottlenecked system is like adding more cars to a traffic jam. That is when it clicked: Backend systems don’t usually fail because of flawed logic. They fail because of an uncontrolled flow of state.
This is a breakdown of how I stripped that system down and rebuilt it using distributed throttling, async queues, and backpressure, shifting my mindset from "How do I make this faster?" to "How do I control the chaos?"
The Core Problem: Uncontrolled Concurrency
The naive architecture suffered from three fatal flaws:
No Rate Limiting: The API accepted payloads as aggressively as clients could fire them.
Synchronous Execution: Every request attempted to execute immediately, tying up the API thread while waiting for slow external servers.
Global Blindness: SMTP servers apply strict rate limits per domain. Firing 50 concurrent requests to Gmail from a single IP is an instant ban, but my system treated all traffic equally.
I needed an architecture that respected the physical limitations of the external world.
Layer 1: Fortifying the Gates (API Limits & Security)
The first rule of distributed systems is that you cannot trust the client. If you expose an endpoint without safeguards, it will eventually become a target for resource exhaustion or a proxy for abusive traffic.
I implemented strict endpoint-specific rate limiting using Redis. A standard /verify endpoint had controlled throughput, while the /bulk endpoint was heavily restricted.
Before a payload even touched the processing queue, it faced strict input validation. Malformed requests and oversized payloads were dropped instantly. This wasn't just about security; it was about protecting compute. Every bad request dropped at the API layer is an expensive, doomed SMTP call avoided downstream.
Layer 2: Decoupling with Queues
Instead of forcing the API to hold the connection open during verification, I decoupled the ingestion from the execution.
The flow became: API → Redis Queue → Async Workers
The API now instantly returns a 202 Accepted status, and the workers pull tasks at their own pace. This completely smoothed out unpredictable traffic spikes and prevented the API layer from blocking.
Layer 3: Distributed Domain Throttling
This was the hardest—and most critical—engineering hurdle.
SMTP servers are adversarial. They absolutely hate multiple concurrent connections from the same IP. To survive with a single IP, I had to implement a distributed domain throttle.
I engineered a Redis-based concurrency control system. Before a worker picks up a task, it checks the domain. If the target is gmail.com, the system ensures a maximum of, say, two concurrent connections globally across all workers. If outlook.com is stricter, it gets throttled to one.
This domain-aware flow control completely eliminated connection drops and greylisting penalties. We stopped getting blocked because we stopped acting like a botnet.
Layer 4: The Safety Valve (Backpressure)
Queues are great, but they are not infinite. If ingestion outpaces processing for too long, the queue blows up your memory.
To prevent this, I introduced backpressure. Before the API accepts a new bulk request, it checks the current depth of the Redis queue. If the system is saturated, the API gracefully rejects the request with a 503 Service Unavailable.
It is always better to reject traffic cleanly at the front door than to let the internal infrastructure collapse under its own weight.
The Takeaway
After implementing these layers, the system stabilized entirely. Throughput became predictable, false negatives vanished, and the single-IP reputation remained pristine.
The biggest lesson I took away from this build isn't specific to email verification. Whether you are building data scraping pipelines, LLM orchestrators, or distributed microservices, the rule is the same: Performance isn't about raw speed. It is about flow control. Stop trying to force your system to go faster, and start engineering it to survive the friction of the real world.
If your system breaks under load, it’s not a scaling problem — it’s a control problem.
Top comments (0)