How to Test Emails in Mandrill, Sendgrid, and Mailgun

In our previous post, we reviewed Sendgrid, Mandrill, and Mailgun – the most popular email sending services. “Test your emails before actually sending them”: each of them repeats this mantra, as does anyone who deals with email development. But do these services offer enough options for quality email testing? Let’s have a closer look at the checkpoints and see if there is a need for a special testing service.

Sendgrid test

Sendgrid offers sandbox mode and sink email addresses for email developers.

Test the API integration in the sandbox mode. In Sendgrid, it’s a mail_settings parameter, which helps validate email sending requests. Sandbox mode allows checking the parameters of your JSON payload. If any errors are found, it returns their detailed description. You can find more information and examples in Sendgrid documentation. It doesn’t affect any of the real sending parameters.
Check the email sending speed. This option can be useful for preparing large bulk email campaigns. Sendgrid has a “black hole” domain @sink.sendgrid.net. It will accept all the messages and immediately remove them. If you have a dedicated IP address, this type of test won’t affect its warm-up. However, all emails sent to this domain will be counted as billable.

Mandrill test

The testing mode in Mandrill offers a more extensive list of features: test API, test stats, and webhooks testing.

1. With the test API key, you will be able to verify your integration and explore its capabilities. As no actual email sending occurs, it affects neither your monthly limits nor your account reputation.
2. In Mandrill’s test mode, you have access to your test stats. Also, you can view and check the email subject, sender and receiver email address as well as the message content.
3. Another interesting feature is webhooks testing. Mandrill provides you with a list of specific email addresses. You can use them to send a message for testing such events as bounces, rejects, and spam complaints.

You can find out more about Mandrill and see the comparison of Mandrill and Mailtrap features here.

Read more about these options in the Mandrill knowledge base.

Mailgun test

Mailgun features a very simple test mode and an email verification tool.

1. To launch the test mode, you should simply set the o:testmode parameter to true. It will help to test your integration as well. Note that Mailgun charges you for the messages sent in this mode.
2. Email verification tool is a specific feature in Mailgun. It is useful for bulk email campaigns. The tool will check all email addresses in your list to identify invalid ones. It helps to decrease the hard bounce rate, which has a negative influence on your sending reputation. Note that email verification is a separate limit, not included in the email sending volume.

Universal Email Testing Checklist for Developers

Each of the mentioned email services recommends running a series of tests before sending either transactional or bulk emails. See below the list of checks that you can perform for any type of email or framework.

1. Testing the email sending capability. It is actually where you check if your code works.

Check if your script is able to send messages in general. As a rule of thumb, separate production (real) email sending from staging or sandbox testing. You don’t want your tests to reach your customers or affect your domain reputation score.

• Make sure that integrations with third-party services are valid.
• Verify that your integration with the chosen email service works perfectly.
• If you are working with bulk emails, it’s always a good idea to validate your email list. A high volume of bounces, rejections or even spam complaints will have a negative impact on your reputation as a sender.

1. Content testing. Regardless of the method you use to compose your message, there is a chance that your input will differ from the output – the message your user will see in the end.

• Check the text and its formatting. In most cases, you will use HTML to build your template, and add images and/or attachments. If you have such an option, always use both text and HTML. This way, your recipient will be able to read the text and understand the message if the HTML is not rendered for some reason.
• Check dynamic content. Be careful and test it thoroughly. Dynamic content may cover user names and some data from your app, like stats or reports, etc. It is usually sent to multiple users, so it is a common area for mistakes. You need to run many test cases to make sure that the dynamic data is applied and displayed correctly. If you use localization, test it as well. All these points lead to high-volume testing and require a secure and efficient tool.
• Make sure the links are valid.
• If your message contains images, test if they are displayed in most email clients. There is still no unified image rendering standard. We have observed that the largest online stores and cloud services host images on their servers and then link to them.
• Perform responsive design testing. Now people use their smartphones even more than laptops. Make sure your message looks good and clear on different screen sizes. There are numerous free HTML templates available for downloading and customizing. So it might be a good idea to use them instead of coding your own.
• Stay aware of the email size. If your recipient’s internet connection is slow, the large message might be pretty irritating.

1. Spam testing.

• Check the spam score of your email to make sure it won’t be blocked by spam filters. Even user-triggered messages might land in the spam folder. It can be caused by the email content itself. Missing information in headers, inconsistency in HTML and text parts, too many images with less text, etc., usually lead to troubles.
• Check your domain and IP reputation, even if you are using those provided by the email service. In addition, you can share the testing results with your team members or even forward your message to your inbox for preview.

To get the full list of checks, refer to our Email Testing Checklist article.

Test Your Emails Now

Tools and Options for Email Testing

When it comes to email testing, the first thing that probably comes to mind is creating several test accounts with the most popular email clients like Gmail, AppleMail, Outlook, or Thunderbird. This will let you instantly check the delivery speed, see in which folder your message landed as well as review the email content and how it is rendered. Also, using this method, you can open testing emails on your smartphone and/or tablet to see their responsiveness.

Still, it’s important to note that although setting up test accounts with several webmail services and/or email clients can bring you good results, it will work well only if you occasionally need to test new templates. Otherwise, you might become overwhelmed with maintaining all your accounts’ inboxes and cleaning out test emails.

In addition, since this falls under production testing, it will affect your monthly credits and sending reputation.

All in all, it’s safe to say that when you need to test bulk email sending that includes dynamic content and localization, other methods should be explored, preferably ones that involve feature-packed tools.

One such tool is Mailtrap Email Sandbox which is primarily geared towards devs and QAs who want to inspect and debug emails in staging without the risk of spamming recipients in the process.

What’s especially great about Mailtrap Email Sandbox is that it’s capable of running all the necessary tests we mentioned earlier:

• Testing the email sending capability – Once you replace your existing SMTP configuration with the one provided to you by Mailtrap Email Sandbox (separate configuration for each of your virtual inboxes), all the testing emails you send will end up in one of your virtual inboxes. For each of your projects and even different stages of projects, you can have a dedicated virtual inbox if you so wish.
• Content testing – With Mailtrap Email Sandbox, you get to see how popular web browsers render your emails and find any issues with images, links, variables, or content, as well as test how the email looks on different device screen sizes. Source code, text, and raw format previews are also available for users of this tool.
• Spam testing – Spam reports and blacklist reports are provided through Mailtrap Email Sandbox’s spam analysis feature. With the help of these, you can know what your overall spam score is, be alerted about any rules that might be treated as suspicious, and check if your sender IP/domain is present on any commonly used blacklists. Other features of the tool that might come in handy during email testing include HTML/CSS analysis, email forwarding to whitelisted recipients, and insight into detailed tech info.

To start using Email Sandbox, you’ll need to create a Mailtrap account and complete a 5-minute setup process. And, once you are a registered Mailtrap user, besides email testing, you will also be able to deliver emails to recipients from your account using Email API – the email sending side of the Mailtrap Email Delivery Platform.

Try Mailtrap for Free

Email Testing Outcomes

A lot of email sending providers offer quite limited testing capabilities. More importantly, they hardly allow users to separate email sending from development and production environments. Instead, some of them just feature third-party apps, which might help you with testing.

So, if you’re looking for a platform you can test, send, and control emails with in one place and don’t want to pay for multiple services, we would recommend going with Mailtrap. It will serve as a reliable end-to-end sending solution for most developers, QAs, and managers.

Still, regardless of what tool(s)/service(s) you go for and what your testing workflows look like, don’t forget to cover the three main steps:

• Testing of sending functionality
• Testing of content
• Testing for spam

Only with these steps out of the way will you be able to avoid many of the common errors that occur in email sending.

Good luck and till next time!

Thank you for choosing this article! Read Mailtrap blog and discover more about different email testing tools!