Building an AI powered and Serverless meal planner with OpenAI, AWS Step functions, AWS Lambda and CDK

#openai #serverless #stepfunctions #cdk

OpenAI’s generative capabilities offer new possibilities when building applications. Combined with Serverless technologies, we can create applications faster while still maintaining the flexibility to iterate and to improve them over time.

In this article, I will show you how to build an application that sends emails containing generated weekly meal plans from a set of ingredients a user provides. We will use OpenAI’s APIs along with AWS Serverless services: Step Functions, AWS Lambda, and Amazon SES.

We will use NodeJs runtime and typescript for the Lambda code as well as CDK for IaC.

What are we going to build ?

We will create an application that allows users to submit via an API a request containing a set of food ingredients and an email address. It will then, asynchronously, send to the user an email containing a meal plan with detailed recipes for a whole week:

Here is the architecture diagram of the application we are going to build:

The relevant parts of this solution:

We use a step function to orchestrate the invocations of Lambda functions that send requests to OpenAI’s APIs to generate recipes from a prompt as well as an image for each recipe.
We use the S3 bucket to store the generated recipe images. These images are served via a CloudFront distribution.
The generated meal plan is then sent via email using SES. We use the SES templates capability to send personalized emails for each user.
The Rest API gateway has a POST route with an integration to the meal planner Step Function.

TL;DR

You can find the full repository with its deployment pipeline here 👇
GitHub - ziedbentahar/serverless-meal-planner-with-aws-and-openai

Let’s deep dive into the code

☝️ Before starting: In order to use OpenAI APIs, you will need to sign up and to create an API KEY. You can follow this link to get started. The use of this API is not free, however new accounts get free credits (tokens) to start experimenting.

Defining the state machine

The first step of this state machine is to generate a meal plan for a week. The second step involves generating a picture for each recipe and then saving it on a S3 bucket. The processing is done in parallel for each recipe using a Map state, this has the advantage to reduce the overall execution time of the state machine. And finally, the last important step is the sending of the email containing the meal plan:

Which translates to this fluent state machine definition in CDK:

You can find the complete CDK definition of the state machine following this link.

Defining the Lambda functions

1- Generate meal plan Lambda:

The challenging part about this step was to find the best prompt that yields good and consistent results from OpenAI completion API. I used the text-davinci-003 GPT model (also referred as GPT-3.5).

When I tried out different prompts, the suggestions were quite good for producing interesting meal plans given a list of coherent ingredients. I was even able to request a structured result in JSON format ready to be processed by the Lambda function. I also experimented with parameters such as temperature, TopP and max_tokens searching for the sweet spot that gets satisfying results.

This prompt produces the best results given our use case:

Generate a dinner meal plan for the whole week with these ingredients <a comma seperated list of ingredients> and with other random ingredients.
Result must be in json format
Each meal recipe contains a name, a five sentences for instructions and an array of ingredients

And here is the code of the Lambda function that handles the generation of the meal plan:

☝️ Some notes:

As depicted on the diagram above, The OpenAI API key is stored on a secret. In this example we use the AWS parameters and secrets Lambda extension to read the secret value from the Lambda. You can learn more about this Lambda extension here.
Even though the completion API was providing consistent response models in JSON, for some reason, the properties on the JSON object were not having a consistent casing as I was experimenting with the API. Hence the use of the getProperty helper function before returning the result; this function ensures getting a property value from an object regardless of its casing.

2- Generate recipe image Lambda:

This Lambda function is similar to the previous one. We use the recipe name that createCompletion API has generated in order to create an image from it by calling createImage (this API uses DALL-E models for image generation) :

createImage API returns an array of URLs, the size of this array depends on the number of variation of the images we want to generate. In our example we are interested in only one single image. The image URL expires after one hour, here is why we pass it to the upload-recipe-image-to-storage Lambda that has the responsibility to download the image and to store it on a S3 Bucket.

3- Send meal plan email Lambda:

The sending of the email uses SES. But first, the Lambda function prepares a template data containing the necessary elements to generate the email:

On the section below, we will see how to use CDK to create a new SES email identity as well as the email template that is used to send the mal plan.

Note: You can find the CDK definitions of these Lambda functions following this link.

Configuring SES

On this example, we use a domain that is already defined in the Route53 public hosted zone; The SES email identity DNS validation is then seamless. We also create the meal plan email template on this nested stack:

☝️ Note: By default, an SES account is in sandbox mode. You are allowed to send emails only to verified identities and you can only send a limited number of emails per 24-hour period. Follow this link to understand the sandbox mode quotas and how to move out of it.

Integrating the API Gateway with the Step function workflow

Creating the RestApi with the Step Function integration is quite easy with CDK, although a bit verbose:

We need to create a role that grants the Api Gateway to states:StartExecution the Step Function. Each request gets validated with the API gateway JSON schema model validation before the execution of the step function.

Wrapping up

In this post, we have seen how combining OpenAI APIs with serverless architecture can help building AI-powered applications with minimal setup and configuration. The capabilities of both of these two worlds are great enablers for building MVPs and iterating faster.

This application can be improved further by taking into account food restrictions or even by creating an AI powered weekly news letter.

You can find the full source code of this application here:
GitHub - ziedbentahar/serverless-meal-planner-with-aws-and-openai