A while ago, I started looking for a service that would allow me to implement email alias functionality. Until now, I have had it configured on Postfix running on my VPS server. Setting up and maintaining an email service is time-consuming. Tasks such as updating SSL certificates, spam filtering, and antivirus protection require ongoing attention. A poorly configured mail server can easily become a target for bots sending spam and malware.
AWS Simple Email Service
AWS SES is a service that operates on a “pay as you go” model. It is designed for sending both transactional notifications and marketing emails. More information about the service can be found at: https://aws.amazon.com/ses/.
The SES service is available in every AWS account. By default, it is configured in sandbox mode. The sandbox mode has several limitations, the most important are:
- The inability to receive emails from unverified senders
- The ability to deliver emails only to a list of verified email addresses
To start using the production plan, you need to request it through the AWS web console. The request is reviewed by AWS Support, which can adjust the account settings to enable the production plan.
Prerequisites
Domain verification
The primary requirement for running the discussed solution is having a domain correctly verified in the SES service. The domain should have a Verified status.
Additionally, DKIM and DMARC records must be configured.
☝️ If the domain is hosted on Route53, this can be done using Terraform code. Otherwise, all necessary records can be obtained directly from the AWS SES console.
# Verified domain identity
resource "aws_ses_domain_identity" "ses_domain" {
domain = var.ses_domain
}
# DKIM identity for email domain
resource "aws_ses_domain_dkim" "ses_domain" {
domain = aws_ses_domain_identity.ses_domain.domain
}
resource "aws_route53_record" "ses_domain_dkim_record" {
count = 3
zone_id = var.zone_id
name = "${aws_ses_domain_dkim.ses_domain.dkim_tokens[count.index]}._domainkey"
type = "CNAME"
ttl = "600"
records = ["${aws_ses_domain_dkim.ses_domain.dkim_tokens[count.index]}.dkim.amazonses.com"]
}
resource "aws_route53_record" "ses_domain_verification_record" {
zone_id = var.zone_id
name = "_amazonses.${var.ses_domain}"
type = "TXT"
ttl = "600"
records = [aws_ses_domain_identity.ses_domain.verification_token]
}
resource "aws_route53_record" "ses_dmarc_record" {
zone_id = var.zone_id
name = "_dmarc.${var.ses_domain}"
type = "TXT"
ttl = "600"
records = ["v=DMARC1; p=none; rua=mailto:postmaster@${var.ses_domain}"]
}
MX Record
To receive emails from external sources, you need to configure an MX record that points to the SMTP server address of the SES service.
resource "aws_route53_record" "this" {
zone_id = var.zone_id
name = var.domain_name
type = "MX"
ttl = 300
records = "10 inbound-smtp.region.amazonaws.com"
}
The region should be replaced with the one where your SES service is running.
Solution architecture
Email processing will be covered later in this article in details.
The Lambda function receives an event from SES, which contains information about the email alias address and the MessageID. The MessageID is also used as the key name in the S3 bucket.
☝️ By utilizing the S3 bucket, we gain the ability to receive emails > up to 40MB in size. The maximum size of an email that can be sent via the boto3 API is 10MB.
After the email is processed, it is sent via SES to the list of recipients.
Implementation Challenges
The SES service comes with certain limitations. The most significant is the inability to send emails with a FROM
header that has not been verified in the service. This is a major obstacle, as it is often not feasible to verify the email address with each individual entity, especially since many of them are automated systems.
Implemented Workaround
The issue described above was resolved by breaking down the received email into its components. The Lambda function analyzes the structure of the email, extracts all textual elements, images, and attachments, modifies the From
and To
addresses, reassembles the email, and sends it to the alias recipient list.
SES Email receiving
To get the solution working, you need to configure email receiving rules. Refer to: Email Receiving Concepts.
In the discussed solution, an email rule named email-aliases is defined, which contains two rule sets:
- trigger-mail-forwarding
- reject
Email Receiving Rules and Actions
☝️ The position parameter plays a crucial role as it determines the execution order of actions.
The action to save data to S3 must be executed before the Lambda function is triggered. Otherwise, the Lambda function will not receive information about the email stored in the bucket.
Receiving rules examples
resource "aws_ses_receipt_rule_set" "this" {
rule_set_name = var.ruleset_name
}
# Add a header to the email and store it in S3
resource "aws_ses_receipt_rule" "this" {
name = "trigger-mail-forwarding"
rule_set_name = aws_ses_receipt_rule_set.this.rule_set_name
recipients = var.recipients
enabled = true
scan_enabled = true
tls_policy = "Require"
add_header_action {
header_name = "X-SES-Forwarded-By"
header_value = "SES Rule ${aws_ses_receipt_rule_set.this.rule_set_name}"
position = 1
}
s3_action {
bucket_name = var.s3_bucket_name
position = 2
}
lambda_action {
function_arn = var.lambda_function_arn
invocation_type = "Event"
position = 3
}
stop_action {
scope = "RuleSet"
position = 4
}
}
resource "aws_ses_receipt_rule" "reject" {
rule_set_name = aws_ses_receipt_rule_set.this.rule_set_name
name = "reject"
enabled = true
scan_enabled = true
tls_policy = "Require"
bounce_action {
message = "This email address is not accepted by this domain."
sender = var.noreply_email
smtp_reply_code = "550"
position = 1
}
stop_action {
scope = "RuleSet"
position = 2
}
}
Lambda code
The function code is available on my github.
kkrolikowski
/
ses-alias-lambda
Lambda designed to send raw emails through AWS Simple Email Services
AWS SES Email Alias Lambda
This function helps solve a limitation of AWS Simple Email Service: you can’t send emails to external recipients using a FROM address that hasn’t been verified. The function disassembles the email and replaces the FROM
address with an address from a domain verified in the SES service. It then reassembles all the parts and sends the email to the target recipients.
How it works
The Lambda function is invoked by the SES service and receives an event containing information about the sender and the MessageID. The MessageID is required to locate the corresponding object with the raw email message in the S3 bucket. Based on the recipient address received in the event, email alias targets are located in the AWS SSM Parameter Store service.
Environment variables
Variable | Description |
---|---|
EMAIL_BUCKET | S3 bucket name with raw email objects |
ALIAS_MAP_PARAM | SSM parameter path with email alias mappings |
Input
…It‘s distributed under the MIT license.
The mentioned ses-alias Lambda function does not contain complex logic. Its task is to retrieve the email content from an S3 bucket, locate the alias recipients in SSM, reassemble the email, and send it using SES.
Mailparser
The most important part of the code is the parser, which contains the logic for processing raw email files.
from email.parser import BytesParser | |
from email import policy | |
from email.mime.text import MIMEText | |
from email.mime.image import MIMEImage | |
from email.mime.multipart import MIMEMultipart | |
from lib.helpers import mail_address_helper, safe_decode | |
def prepare_email_message(raw_email: bytes, targets: list) -> MIMEMultipart: | |
''' | |
Parse the raw email and prepare a new email message to be sent. | |
raw_email: bytes - The raw email message | |
targets: list - The list of email addresses to send the email to | |
return: MIMEMultipart - The new email message | |
''' | |
# | |
### Objects needed to parse and create email message | |
# | |
# Object to parse incoming email | |
mailObject = BytesParser(policy=policy.default) | |
# Create a MIMEMultipart objects to store each part of the email | |
mainMSG = MIMEMultipart('mixed') # The main message and attachments | |
relatedMSG = MIMEMultipart('related') # HTML and inline images | |
alternativeMSG = MIMEMultipart('alternative') # Plain text part | |
# Parse the raw email from S3 bucket | |
mail = mailObject.parsebytes(raw_email) | |
# Define the content types that are considered binary | |
binary_content_types = ['application', 'audio', 'video'] | |
alias_targets = ",".join(targets) | |
# Create new email headers compatible with SES | |
mainMSG["From"] = mail_address_helper(mail.get("To")) # SES requires the From header to be the verified recipient | |
mainMSG["To"] = alias_targets | |
mainMSG["Reply-To"] = mail_address_helper(mail.get("From")) | |
mainMSG["Subject"] = mail.get("Subject") | |
# Loop through the parts of the email | |
if mail.is_multipart(): | |
for part in mail.walk(): | |
# Skip the part if it has no payload | |
if part.get_payload(decode=True) is None: | |
continue | |
# Obtain the content type and disposition of the part | |
content_type = part.get_content_type() | |
content_disposition = part.get_content_disposition() | |
# Obtain the charset of the part or default to utf-8 | |
charset = part.get_content_charset() or 'utf-8' | |
# html goes to relatedMSG, text goes to alternativeMSG | |
if content_type == 'text/html': | |
html = safe_decode(part.get_payload(decode=True), charset) | |
relatedMSG.attach(MIMEText(html, content_type.split('/')[1])) | |
elif content_type == 'text/plain': | |
text = safe_decode(part.get_payload(decode=True), charset) | |
alternativeMSG.attach(MIMEText(text, content_type.split('/')[1])) | |
# Inline images go to relatedMSG | |
elif content_disposition == 'inline' and content_type.startswith('image/'): | |
filename = part.get_filename() | |
content_id = part.get("Content-ID") | |
if not content_id: | |
# Generate a unique Content-ID if it is missing | |
content_id = filename or "image-" + str(hash(part.get_payload(decode=True))) | |
# Remove the angle brackets from the Content-ID | |
content_id = content_id.strip("<>") | |
# Create a new MIMEImage object for decoded image | |
image = MIMEImage(part.get_payload(decode=True), _subtype=content_type.split('/')[1]) | |
# Add the Content-ID and Content-Disposition headers, thy are required for inline images | |
image.add_header('Content-ID', f'<{content_id}>') | |
image.add_header('Content-Disposition', 'inline', filename=filename) | |
# Attach the image to the relatedMSG | |
relatedMSG.attach(image) | |
# Add attachments to the new email message | |
elif content_disposition == 'attachment' or any([x in content_type for x in binary_content_types]): | |
filename = part.get_filename() | |
if filename: | |
part.replace_header('Content-Disposition', f'attachment; filename="{filename}"') | |
part.set_type('application/octet-stream') | |
mainMSG.attach(part) | |
else: | |
# If the email is not multipart, add the text to the alternative message | |
text = safe_decode(mail.get_payload(decode=True), mail.get_content_charset() or 'utf-8') | |
if 'doctype html' in text.lower(): | |
relatedMSG.attach(MIMEText(text, 'html')) | |
else: | |
alternativeMSG.attach(MIMEText(text, 'plain')) | |
# Assemble the email message parts | |
alternativeMSG.attach(relatedMSG) | |
mainMSG.attach(alternativeMSG) | |
return mainMSG |
MIME Standard
Understanding the parser's functionality requires familiarity with the structure of an email message. The most common emails today are those containing more or less complex HTML code with embedded images. Additionally, there are also attachments.
To ensure such messages can be correctly read by client software, the MIME standard was introduced: Multipurpose Internet Mail Extensions.
Read more about MIME in RFC 2046
The first document on this topic was published in 1996, long before XML and JSON standards became popular and dominated the internet.
To better illustrate what we are dealing with, I have prepared the following diagram of an email message in the multipart MIME format.
Of course, not all emails we encounter are of this type; there can also be messages containing only plain text or only HTML. However, there's nothing preventing such content from also being placed into an appropriate MIME-type object.
Conclusion
The AWS Simple Email Service can be used to build functionality for handling email aliases.
Costs
With a low email volume, it will cost very little. In terms of expenses, it is an attractive alternative to other email solutions available on the market.
S3 bucket utilization: each object represents one processed email.
Limitations
AWS Simple Email Service imposes restrictions on sending emails to external recipients from domains (or email addresses) that have not been verified within the service managed under the account. Even when using the production plan, this limitation still applies.
As a result, emails must be sent with a modified "From" address, which necessitates implementing some form of compromise.
Another limitation is the size of the emails supported:
- 10MB: The maximum size of a single email that can be sent via Lambda, as limited by the SES API.
- 40MB: The maximum size of a single email that AWS SES can save to an S3 bucket.
These are relatively generous limits for standard notification emails, but if you plan to send larger attachments or images, these limits need to be taken into account.
Top comments (0)