AWS SES -> Gmail using Terraform

#aws #terraform #route53 #ses

The goal

Let's say you have your own domain and you want to have an email address in it, but you don't want to mess with the setup and maintenance of an email server. You also have a working personal email address at Gmail or somewhere else. There is a solution for you.

The architecture

AWS Simple Email Service (SES) is the thing we can leverage to achieve this goal. The basic diagram that represents the solution is here:

So there are a few steps in this solution:

A message is sent to your custom email address
DNS service, AWS Route53 in our case, has an MX entry that points to SES
AWS SES sends this email to S3 bucket
S3 bucket triggers Lambda that reads the message and forwards it to your Gmail address
Once you get this email in your Gmail mailbox you're able to response from your custom email address

The implementation

Since we're using AWS, it makes sense to leverage an IAC approach and write Terraform configuration for all the components involved in this solution. Check here if you want to see the final solution right away.

First of all, make sure you have your bucket for storing your Terraform state set there:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }
  backend "s3" {
    bucket = "<put your bucket here>"
    key    = "ses-gmail-custom-domain/terraform.tfstate"
    region = "us-east-1"
  }
}

Then, let's create an SES Identity and DKIM setup:

## SES domain identity
resource "aws_ses_domain_identity" "identity" {
  domain = var.domain_name
}

## SES domain identity DKIM setup
resource "aws_ses_domain_dkim" "dkim" {
  domain = aws_ses_domain_identity.identity.domain
}

Once done, let's verify domain ownership, set MX, DKIM, SPF, DMARC records in Route53:

## SES domain identity -> custom domain verification
resource "aws_route53_record" "amazonses_identity_verification_record" {
  zone_id = data.aws_route53_zone.hosted_zone.id
  name    = "_amazonses.${var.domain_name}"
  type    = "TXT"
  ttl     = "600"
  records = [aws_ses_domain_identity.identity.verification_token]
}

## SES domain identity DKIM entries that allow recipient's mail server to verify digital signature included in email by using a public key stored in these entries
resource "aws_route53_record" "amazonses_identity_dkim_records" {
  for_each = toset(aws_ses_domain_dkim.dkim.dkim_tokens)

  zone_id = data.aws_route53_zone.hosted_zone.id
  name    = "${each.key}._domainkey.${aws_ses_domain_identity.identity.domain}"
  type    = "CNAME"
  ttl     = 600
  records = ["${each.key}.dkim.amazonses.com"]
}

## MX record pointed to SES server
resource "aws_route53_record" "mx_record" {
  zone_id = data.aws_route53_zone.hosted_zone.id
  name    = data.aws_route53_zone.hosted_zone.name
  type    = "MX"
  ttl     = 600
  records = ["10 inbound-smtp.${var.region}.amazonaws.com"]
}

## SPF record that shows that amazonses.com is allowed to send email on behalf of our custom domain
resource "aws_route53_record" "spf_record" {
  zone_id = data.aws_route53_zone.hosted_zone.id
  name    = data.aws_route53_zone.hosted_zone.name
  type    = "TXT"
  ttl     = 600
  records = ["v=spf1 include:amazonses.com -all"]
}

## DMARC record
resource "aws_route53_record" "dmarc_record" {
  zone_id = data.aws_route53_zone.hosted_zone.id
  name    = "_dmarc.${data.aws_route53_zone.hosted_zone.name}"
  type    = "TXT"
  ttl     = 600
  records = ["v=DMARC1; p=none;"]
}

Then, we can create an S3 bucket that will be used to store the emails sent to our custom domain and trigger the lambda funtion to send them:

## S3 bucket for incoming email
resource "aws_s3_bucket" "incoming_email_bucket" {
  bucket = "ses-inbox-${var.author}"
}

resource "aws_s3_bucket_server_side_encryption_configuration" "encryption" {
  bucket = aws_s3_bucket.incoming_email_bucket.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

resource "aws_s3_bucket_notification" "bucket_notification" {
  bucket = aws_s3_bucket.incoming_email_bucket.id

  lambda_function {
    lambda_function_arn = aws_lambda_function.lambda_forwarder.arn
    events              = ["s3:ObjectCreated:*"]
  }

  depends_on = [aws_lambda_permission.s3_bucket_event]
}

Every new email in the bucket will trigger a lambda function that pulls this message from the bucket and sends it to our Gmail account using SES (don't forget about the permissions):

## Lambda email forwarder
data "archive_file" "lambda_zip" {
  output_path = "lambda.zip"
  type        = "zip"
  source_dir  = "lambda-forwarder-source-code"
}

resource "aws_iam_role" "lambda_role" {
  name_prefix        = "ses-email-forwarder-role"
  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}
EOF
}

resource "aws_iam_role_policy" "lambda_policy" {
  name_prefix = "ses-email-forwarder-role-policy"
  policy = jsonencode(
    {
      Version : "2012-10-17",
      Statement : [
        {
          "Effect" : "Allow",
          "Action" : [
            "logs:CreateLogGroup",
            "logs:CreateLogStream",
            "logs:PutLogEvents",
            "logs:DescribeLogStreams"
            ], Resource : [
            "*"
          ]
        },
        {
          "Effect" : "Allow",
          "Action" : [
            "s3:GetObject"
          ],
          "Resource" : [
            "${aws_s3_bucket.incoming_email_bucket.arn}/*"
          ]
        },
        {
          "Effect" : "Allow",
          "Action" : [
            "ses:SendEmail",
            "ses:SendRawEmail"
          ],
          "Resource" : "*"
        }
      ]
  })
  role = aws_iam_role.lambda_role.id
}

resource "aws_lambda_function" "lambda_forwarder" {
  filename         = "lambda.zip"
  function_name    = "ses-email-forwarder"
  role             = aws_iam_role.lambda_role.arn
  handler          = "lambda.lambda_handler"
  source_code_hash = data.archive_file.lambda_zip.output_base64sha256
  runtime          = "python3.9"
  environment {
    variables = {
      REGION     = var.region
      FORWARD_TO = var.original_email
      DOMAIN     = var.domain_name
    }
  }
}

resource "aws_lambda_permission" "s3_bucket_event" {
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.lambda_forwarder.arn
  principal     = "s3.amazonaws.com"
  statement_id  = "event_permissions_from_${aws_s3_bucket.incoming_email_bucket.bucket}"
  source_arn    = aws_s3_bucket.incoming_email_bucket.arn
}

The full Lambda source code could be found in the git repo with some other things like role policies and roles that are being tied to the objects above, not going to go too deep into it here.

So, once the Terraform configuration is applied, the output will contain parameters we need for setting up the Gmail part:

output "ses_smtp_server" {
  value = "email-smtp.${var.region}.amazonaws.com" ## check in your SES -> SMTP settings
}

output "ses_smtp_server_port" {
  value = "587"
}

## SMTP user credentials that would be used on Gmail side to send emails on SES behalf
output "ses_smtp_user_username" {
  value = nonsensitive(aws_iam_access_key.ses_smtp_access_key.id) # just for testing purposes, shouldn't be done in prod
}

output "ses_smtp_user_password" {
  value = nonsensitive(aws_iam_access_key.ses_smtp_access_key.ses_smtp_password_v4) # just for testing purposes, shouldn't be done in prod
}

Having them, we can go to Gmail account settings and create an alias with a new custom domain name.

One important notice about SES Sandbox mode

To be able to send and reply to every email address you will need to jump out of the sandbox mode in SES. To do so, raise a ticket with AWS support and say something like this:

I've set up a custom domain in SES that's being tied to my gmail.com account. So, when someone sends email to my custom email it goes to my gmail.com email. But I can't respond to the recipient from directly, since in the sandbox mode I have to verify the recipient's email first. So, to avoid this limitation, I'd like to jump out of the sandbox mode and be able to reply to anyone who's reaching out to me via my custom email .

Once approved you'll see something like this: