At Knock, we get asked "which email provider should I use?" all the time.
Until now, comparing email provider performance has been harder than it should be. The existing resources are mostly vibes—"Best of" articles, anonymous Reddit threads, or LLM recommendations trained on that same content. Objective performance data is always missing.
We believe evaluating a vendor should look the same as profiling a production system: data collected at scale, aggregated over time, under real-world conditions. So we turned to our own data. Every year, Knock sends billions of API requests to downstream email providers.
With this data, we built email API benchmarks, a real-time dashboard that aggregates performance metrics across 10 major email providers. Here's what we learned.
API response time: speed matters for transactional email
Most email APIs had a median response time of less than 500ms, which is fast enough for most use cases. But the variance between providers tells a more interesting story.
The three fastest providers on average were:
- SendGrid at 22ms (p50)
- Postmark at 33ms (p50)
- Resend at 79ms (p50)
Response time becomes critical for time-sensitive notifications like one-time passwords, fraud alerts, and two-factor authentication codes. At the p99 percentile, the long tail of requests where performance degrades, SendGrid maintained an average of 157ms, outperforming other providers.
We measure response time as the time to receive a 200 from a provider, computed across multiple percentiles (p50, p90, p95, and p99) using exact quantile calculations in ClickHouse. This includes HTTP connection pooling, latency, service time, and any automatic retries, making it a useful indicator of system resilience.
Error rates: reliability under real-world conditions
Most email APIs have an average daily error rate close to 0.00%, which sounds good until you look at peak error rates. During our 90-day observation window, peak daily error rates were grouped between 0.02% and 3.41%, which is a significant spread.
AWS SES stood out here, with the most days showing error rates below 0.01%, covering the entire 90-day window. This consistency matters when you're sending millions of transactional emails and can't afford degraded delivery.
We track error rate as the ratio of 5XX responses to total requests for each provider over the period. Since Knock's retry logic treats most 5XX codes as retriable, a single request can produce multiple error entries if the full backoff sequence runs—meaning our error rates may be higher than what you'd see without retry logic (but your emails get sent 😉).
Provider adoption trends
Looking at both sending volume and channel growth on Knock's platform revealed some interesting trends:
- SendGrid dominated by volume, with users sending over 500 million messages through the service in the last 90 days
- Resend led in new channel connections, continuing its upward trend in adoption
This split between volume leaders and growth leaders suggests the market is evolving. Established providers maintain massive install bases, while newer developer-focused alternatives are capturing new projects.
We rank providers from one to ten based on new channel connections created during the period, then compare the first and second halves of the window to detect directional trends.
Status page correlation: incident transparency matters
On each provider detail page, we integrate with the provider's public status page via RSS or API and overlay reported incidents on the performance data we collect.
Our view is that observed degradation should correspond with a status page entry. The degree of correlation between measured performance and public incidents is a useful signal for how a provider handles incident management. If you've ever been on-call, this is something that matters when you're debugging production issues at 2am.
Pricing at scale
We've modeled each provider's price curve so you can see how costs scale with volume. The comparison view puts two providers side by side, making it straightforward to weigh pricing alongside performance when evaluating a switch or new integration.
At 5 million emails per month, pricing differences become significant. AWS SES remains the lowest-cost option at scale, while providers like SendGrid, Postmark, and Resend price higher but include additional features and support.
What's next for email API benchmarks
We launched with this subset of metrics because they were the most useful and the easiest for us to obtain. We're already exploring the best way to include deliverability and time-to-inbox metrics.
Knock collects a wide variety of downstream message events, like opens, reads, and link clicks. We'd love to hear from the developer community about what would be valuable to see in a resource like this.
Try it yourself
If you're evaluating email providers today, explore the benchmarks and see how your current provider stacks up.
And if you're looking for a notification platform that gives you the flexibility to use any email provider—or switch between them without changing your code—sign up for Knock for free.
What email provider are you currently using? Have you experienced the performance differences we measured? Drop a comment below with your experiences.
Top comments (1)
Great resource. Nice work, Jeff.