Reverse ETL: Building a Bridge from Your Data Warehouse to SaaS Applications
Your data warehouse is full of valuable insights, but they're useless if they stay locked behind analytics tools. Reverse ETL platforms solve this by automatically syncing processed data back to where your business teams actually work: Salesforce, HubSpot, Intercom, and dozens of other SaaS applications. This architecture challenges traditional data pipelines and introduces fascinating design problems around rate limiting, consistency, and scale.
Architecture Overview
A reverse ETL platform typically consists of five core layers working in concert. The extraction layer reads from your data warehouse, running scheduled queries or listening to change streams to identify which records need syncing. The transformation layer maps warehouse schemas to SaaS API formats, handling field translations and data type conversions. The orchestration layer manages job scheduling, retry logic, and state tracking, ensuring each sync operation is idempotent and traceable. The delivery layer actually sends data to target APIs, while the monitoring layer tracks success rates, latency, and errors across all integrations.
What makes this architecture interesting is its asymmetry compared to traditional ETL. Instead of pulling data on demand, a reverse ETL system must push data reliably to systems it doesn't fully control. The warehouse is stable and fast, but external APIs have unpredictable behavior. This means the architecture must be built around resilience, with heavy emphasis on queuing, batching, and graceful degradation.
The orchestration layer is where much of the intelligence lives. Rather than naive parallelization, it implements adaptive batching strategies, learns from past API behavior, and maintains detailed records of what's been synced to each destination. This stateful approach is critical because you can't afford to sync the same record twice or lose track of which customers got updated.
Handling Rate Limits at Scale
Here's where the follow-up question gets really interesting: what happens when you need to sync 50,000 customer records to Salesforce, but the API only allows 300 requests per minute? Naive approaches fail fast. A well-designed system implements several layers of protection. First, the orchestration engine maintains per-destination rate limit budgets, calculated from API documentation plus empirical observations. Second, it implements exponential backoff with jitter, so retries don't create thundering herds. Third, it uses token bucket or sliding window algorithms to smooth request distribution, rather than sending bursts followed by idle periods.
But the real sophistication comes from predictive backpressure. The system monitors each API's actual response times and error rates, adjusting its throughput dynamically. If Salesforce starts returning 429 (Too Many Requests) responses, the system doesn't just retry blindly, it immediately reduces the request rate across the entire cluster, preventing cascading failures. Additionally, smaller batches are queued with higher priority, ensuring that updates to critical records (new high-value customers, for example) get delivered faster than bulk syncs of less time-sensitive data.
Watch the Full Design Process
Want to see how this architecture comes together? I recently designed this system from scratch using InfraSketch, and captured the entire process. You'll see how the components evolve, how constraints shape decisions, and how rate limiting logic fits into the bigger picture.
Try It Yourself
This is Day 98 of the 365-day system design challenge, and one thing I've learned is that the best way to internalize architecture patterns is to design them yourself. Head over to InfraSketch and describe your system in plain English. In seconds, you'll have a professional architecture diagram, complete with a design document. Try describing a reverse ETL system, a data pipeline, or any system that's been on your mind. You might surprise yourself with what you discover.
Top comments (0)