Sending emails only once with AWS Step Functions

Most email delivery API:s will send two emails if you send two requests. This might be a problem since there are many ways a requests can be duplicated. Probably you don't want to send more emails than necessary. For example both SQS and EventBridge guarantees at least once delivery.

Let’s take a look at how we can use Step Functions and DynamoDB to avoid sending multiple emails. We will use Condition Expressions in DynamoDB to see if we have already processed an email. The basic flow of the state machine is:

• Create a hash based on the email.
• Try to save the hash in DynamoDB. If the item already exists, abort and do nothing.
• If the item did not exist, proceed and send the email.

Multiple executions with the same input will not send additional emails. We have idempotency.

Create the hash

We want to create a short string representation of an email and save it in a database. Let’s use a hash function that creates a unique string based on sender, receiver, subject, and content. This will be the unique key in the database, which we will use to determine whether we have processed the email before or not.

const crypto = require('crypto');

exports.handler = async (email) => {
    const combined = `${email.from}${email.to}${email.subject}${email.content}`
    const hash = crypto.createHash('sha256').update(combined).digest('base64');
    return hash;
};

Save the item

We will then try to insert the hash in our database, with a Condition Expression: The item must not already exist. If the item already exists - indicating that we have already processed it - we want to catch this error and not send the email. If it doesn’t exist we proceed to send the email.

Send the email

There are many ways to send an email. Implement according to your email delivery API. Note that this step can be replaced with pretty much anything that you want to do only once. For example payments and orders.

exports.handler = async (email) => {
    // TODO implement send email
    return "sent email";
};

If the email has not been processed before, the execution should look like this:

Email was already processed

Catch the ConditionalCheckFailedException and use the Pass state in Step Functions. It's important to end in success state even though we actually didn't send the email this time. This is a characteristic of an idempotent API. Use the Fail state for all other errors.

If the email has already been sent, the execution should look like this:

Conclusion

That's pretty much it! The definition for the state machine is available here: https://gist.github.com/danielfyhr/4144dba260cc2bce1509d12cfd998664

Standard Workflows guarantee exactly once execution of each workflow step: https://aws.amazon.com/step-functions/faqs/

Comments

[Wojciech Matuszewski]

Thank you for sharing.

One question: Why did you not leverage the name attribute on the StepFunction execution to enforce uniqueness?

According to the StepFunctions API documentation the name has to be unique (90 days grace period). Maybe you could use that instead of saving the hash into the DB?

[Daniel Fyhr]

Thank you for your feedback.

I think your solution with using the name is a good way to solve it. In that case maybe you can use an outer state machine to create the hash and then start the inner machine (using the hash as name).

Some things that come to mind as advantages with using DynamoDB are if you want more control over the grace period by using time to live in DynamoDB. Another use case is if you want to keep track of what you have sent without relying on the email delivery provider. In that case you could use something like

{
  PK: `EMAIL#${email.to}`,
  SK: `HASH#${hash}`,
  // additional fields
}

To be able to query on receiving email address.

Do you have any other aspects to consider?