DEV Community: Brenon Araujo

AWS Serverless Text-to-Speech Solution with AWS CDK and TypeScript pt.II

Brenon Araujo — Wed, 12 Jul 2023 05:44:44 +0000

In the first part of this article check it out right here, we saw an overview of the solution. Now, we will delve into the code and explain how to build the solution from scratch. I will highlight the key aspects of the codebase that I have created for exploring this tool and constructing the serverless solution.

If you haven't read the first part yet, I recommend doing so before proceeding. However, if you are eager to get your hands dirty and build this solution step by step, you can continue reading.

Alternatively, if you simply want to deploy and test how this solution works, you can follow the instructions provided in the readme of the GitHub repository

Build from scratch

To create this application you'll need to have Node.js (>= 16.3.0), AWS CDK, and AWS CLI installed. You'll also need an AWS account. The AWS CLI should be configured with your AWS credentials.

If you try to run CDK with a Node version below the one indicated above, you will see the warning message: “v16.2.0 is unsupported and has known compatibility issues with this software.” Update your Node version, and it should be fine.

Here are the relevant links to the official AWS documentation that will help you with the pre-requisites:

AWS CDK: The AWS CDK Developer Guide provides detailed information on how to install and configure AWS CDK. The AWS CDK (Cloud Development Kit) can be installed via npm (Node Package Manager) using the command: npm install -g aws-cdk
AWS CLI: You can install the AWS CLI (Command Line Interface) following the instructions in the AWS CLI User Guide. You can install the AWS CLI using pip, which is a package manager for Python. The command to install the AWS CLI is: pip install awscli
Configuring the AWS CLI: Once the AWS CLI is installed, you will need to configure it with your AWS credentials. This includes your Access Key ID and Secret Access Key. Instructions for this can be found in the AWS CLI User Guide. The command to configure the AWS CLI is: aws configure

Step 1: Create a new CDK project

Create a new directory for the project:

mkdir text-to-speech-app
cd text-to-speech-app

Initialize a new CDK project:

cdk init --language=typescript

Step 2: Install AWS CDK packages

You will need several packages for this application. Install them as follows:

npm install @aws-cdk/aws-s3 @aws-cdk/aws-dynamodb @aws-cdk/aws-sqs @aws-cdk/aws-lambda @aws-cdk/aws-lambda-event-sources @aws-cdk/aws-iam --save

After that your package json has to look like this:

"dependencies": { "@aws-cdk/aws-dynamodb": "^1.204.0", "@aws-cdk/aws-iam": "^1.204.0", "@aws-cdk/aws-lambda": "^1.204.0", "@aws-cdk/aws-lambda-event-sources": "^1.204.0", "@aws-cdk/aws-s3": "^1.204.0", "@aws-cdk/aws-s3-notifications": "^1.204.0", "@aws-cdk/aws-sqs": "^1.204.0", "aws-cdk-lib": "2.87.0", "constructs": "^10.0.0", "source-map-support": "^0.5.21" }

Step 3: Create S3 Buckets

Open the lib/text-to-speech-app-stack.ts file and start creating the S3 buckets:

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';

import * as s3 from 'aws-cdk-lib/aws-s3';

export class TextToSpeechAppStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const textBucket = new s3.Bucket(this, 'TextBucket', {
      versioned: false,
      encryption: s3.BucketEncryption.S3_MANAGED,
      blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
    });

    const audioBucket = new s3.Bucket(this, 'AudioBucket', {
      versioned: false,
      encryption: s3.BucketEncryption.S3_MANAGED,
      blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
    });
  }
}

Step 4: Create a DynamoDB Table

Add the following lines to create the MetadataTable DynamoDB table:

import * as dynamodb from 'aws-cdk-lib/aws-dynamodb';

// inside the constructor bellow audioBucket Creation

const metadataTable = new dynamodb.Table(this, 'MetadataTable', {
      partitionKey: { name: 'uuid', type: dynamodb.AttributeType.STRING },
      sortKey: { name: 'submissionTime', type: dynamodb.AttributeType.STRING },
      billingMode: dynamodb.BillingMode.PAY_PER_REQUEST,
    });

Step 5: Create an SQS Queue

Next, create the ProcessingQueue SQS queue:

import * as sqs from 'aws-cdk-lib/aws-sqs';

// inside the constructor bellow Metadatatable

  const processingQueue = new sqs.Queue(this, 'ProcessingQueue', {
      visibilityTimeout: cdk.Duration.seconds(300), // 5 minutes
    });

Step 6: Create an S3 Event Notification with SQS as target

As we had our textBucket and processingQueue now we can create an S3 Event Notification to sends a message to our processing queue when a file is uploaded to the /upload path.

//
import * as s3n from 'aws-cdk-lib/aws-s3-notifications';

// inside the constructor bellow processingQueue    textBucket.addEventNotification(s3.EventType.OBJECT_CREATED, new s3n.SqsDestination(processingQueue), { prefix: 'upload/' });

We've created the main infrastructure to be used in our solution, and the highlight here is that we've written this in only 38 lines of TypeScript code.

You can execute the cdk deploy command to deploy the current state of our Stack before we add the function as constructs to be used in this stack.

The CDK shows to you the progress of deploy right on your terminal:

And after this process had finished you can go to your console to check the status of our infrastructure.

Now we can go forward and create the lambda function that will actually does the magic for us

Step 7: Create the TextToSpeech Construct

Our lambda function will be created as a new constructor in our CDK project, for this let's create a new folder on the /lib called TextToSpeech and inside this folder lets creates two new files TextToSpeech.ts and TextToSpeech.lambda.ts, our project structure will be like this:

Now we can go to the TextToSpeech.ts file and create our lambda constructor, here we'll add all the lambda function configurations:

import { Construct } from "constructs";

import { IBucket } from "aws-cdk-lib/aws-s3";
import { ITable } from "aws-cdk-lib/aws-dynamodb";
import { IQueue } from "aws-cdk-lib/aws-sqs";

export class TextToSpeech extends Construct {
    constructor(scope: Construct, id: string, props: {
        textBucket: IBucket,
        audioBucket: IBucket,
        metadataTable: ITable,
        processingQueue: IQueue

    }) {
        super(scope, id)

       }
}

This is our base constructor where we'll define all the lambda resources, the highlight here is the ‘props’ where we've declared all the necessary resources the lambda will interact with.

Step 8: Now we can finally define our lambda funtion

import path = require("path");
import * as cdk from 'aws-cdk-lib';

import { NodejsFunction } from "aws-cdk-lib/aws-lambda-nodejs";
import * as iam from 'aws-cdk-lib/aws-iam';
import { IBucket } from "aws-cdk-lib/aws-s3";
import { ITable } from "aws-cdk-lib/aws-dynamodb";
import { SqsEventSource } from 'aws-cdk-lib/aws-lambda-event-sources';
import { IQueue } from "aws-cdk-lib/aws-sqs";

// Inside the constructor
const textToSpeechFunction = new NodejsFunction(this, "NodejsFunction", {
            entry: path.resolve(__dirname, "TextToSpeech.lambda.ts"),
            bundling: {
                nodeModules: ['aws-sdk'],
            },
            environment: {
                TEXT_BUCKET_NAME: props.textBucket.bucketName,
                AUDIO_BUCKET_NAME: props.audioBucket.bucketName,
                METADATA_TABLE_NAME: props.metadataTable.tableName
            },
            timeout: cdk.Duration.seconds(60) // 1 min
        })

The highlight here is that we use path to define where the lambda code is in the entry propertie and from now we just pass in the bundling propertie the aws-sdk depedencie to be applied at our lambda bundle.

The timeout on our tests was enoght to use this API that has an affordable response time.

Step 9: Grant Permissions to the Resources

Now we need to grant the necessary permissions to our resources.

props.textBucket.grantRead(textToSpeechFunction);
        props.metadataTable.grantReadWriteData(textToSpeechFunction);
        props.audioBucket.grantWrite(textToSpeechFunction);
        textToSpeechFunction.addToRolePolicy(
            new iam.PolicyStatement({
                actions: ['polly:SynthesizeSpeech'],
                resources: ['*'],
            })
        );

        textToSpeechFunction.addEventSource(
            new SqsEventSource(props.processingQueue)
        )
);

In this instance, we utilize the resources that we've declared in the constructor props to configure the necessary permissions. This piece of code simplifies the creation of roles and policies by using the grantRead and grantReadWriteData functions. Following this, we use the IAM PolicyStatement to add a custom policy to our Lambda function, specifically defining the 'synthesizeSpeech' action.

The full file can be founded here at github

Step 10: Create the Lambda Function Code

Now go to the TextToSpeech.lambda.ts file to define our lambda code as bellow:

import { S3, DynamoDB, Polly } from 'aws-sdk';
import { SQSEvent } from 'aws-lambda';
import { GenerateUUID } from './utils/UUIDGenerator';

const s3 = new S3();
const dynamodb = new DynamoDB.DocumentClient();
const polly = new Polly();

exports.handler = async (event: SQSEvent) => {
    const textBucketName = process.env.TEXT_BUCKET_NAME || '';
    const audioBucketName = process.env.AUDIO_BUCKET_NAME || '';
    const metadataTableName = process.env.METADATA_TABLE_NAME || '';

    for (const record of event.Records) {
        const body = JSON.parse(record.body);
        const textKey = body.Records[0].s3.object.key;

        // Read the text file from the text bucket
        const textObject = await s3.getObject({
            Bucket: textBucketName,
            Key: textKey
        }).promise();
        if (!textObject.Body) {
            throw new Error(`Failed to get text body from S3 object: ${textKey}`);
        }
        const text = textObject.Body.toString();

        const pollyResponse = await polly.synthesizeSpeech({
            OutputFormat: 'mp3',
            Text: text,
            VoiceId: 'Joanna',
        }).promise();

        if (pollyResponse.AudioStream) {
            const date = new Date();
            const year = date.getFullYear();
            const month = date.getMonth() + 1;
            const day = date.getDate();
            const timestamp = Date.now();

            const uuid = GenerateUUID();

            const audioKey = `synthesized/${uuid}/year=${year}/month=${month}/day=${day}/${timestamp}.mp3`;
            await s3.putObject({
                Bucket: audioBucketName,
                Key: audioKey,
                Body: pollyResponse.AudioStream
            }).promise();

            await dynamodb.put({
                TableName: metadataTableName,
                Item: {
                    'uuid': uuid,
                    'submissionTime': textObject.LastModified?.toISOString(),
                    'textKey': textKey,
                    'audioKey': audioKey,
                    'characters': pollyResponse.RequestCharacters,
                    'status': 'completed'
                }
            }).promise();

            console.log(`Process ${uuid} synthesized file ${textKey} with ${pollyResponse.RequestCharacters}, saved at: ${audioKey}`)
        }
    }

    return {
        statusCode: 200,
        body: JSON.stringify('Text to speech conversion completed successfully!'),
    };
};

Not any news right here it's just our implementation to handle the sqs event grab the s3 objectKey and than using the Amazon Polly from aws-sdk to synthesize the text from file into audio. It saves the data into the metadata table when finishes

Step 11: Uses the TextToSpeech constructor

Now we just have to import our newly created constructor into our stack file text-to-speech-app-stack.ts

import { TextToSpeech } from './TextToSpeech/TextToSpeech';

// Bellow the addEventNotification 

    new TextToSpeech(this, 'TextToSpeech', {
      textBucket: textBucket,
      audioBucket: audioBucket,
      metadataTable: metadataTable,
      processingQueue: processingQueue
    });

After this setup we can just deploy our Stack and test if its everything working.

Step 12: Deploy the Application

Finally, build and deploy the application:

npm run build
cdk deploy

This will build your application and deploy it to your AWS account.

Testing our App

Once the deployment process is complete, please access your account and locate the text bucket. Place a text file into the bucket. After a short period of time, you will be able to view your audio file in the audio bucket.

To view detailed information about each processed file and the number of characters used to synthesize the texts you have sent, refer to the metadata table.

If you prefer, you can directly download the audio files from the S3 platform. Alternatively, you have the option to explore the journey and integrate an API with CDK. I highly recommend this option as it is simple to implement.

Final Thoughts and Further Exploration

After exploring this Text-to-speech solution, I am excited about the potential to create more solutions using CDK and TypeScript. TypeScript is particularly useful because it is a typed language, which aligns well with backend and frontend development and now to infrastructure development too. CDK also offers the opportunity to use the same language for infrastructure code. Additionally, I am interested in exploring CDK in other programming languages. In my opinion, CDK simplifies a lot of the previous challenges we faced with tools like terraform or cloudFormation, which the CDK utilizes behind the scenes.

If you have any thoughts or would like to discuss further, please let me know. I also welcome any feedback. Thank you for joining me on this journey, and stay tuned for potential new posts on my pages.

AWS Serverless Text-to-Speech Solution with AWS CDK and TypeScript pt.I

Brenon Araujo — Wed, 12 Jul 2023 05:43:16 +0000

Learn how to use AWS serverless tools to synthesize text into human-like speech audio files, all written in TypeScript and deployed with AWS CDK.

In the world of cloud development, 'serverless' is a magical term. It not only simplifies application architecture (though this claim can be contentious), but it also significantly reduces operational costs. Today, I'm going to show you how to build a serverless Text-to-Speech pipeline using the AWS Cloud Development Kit (CDK), written in TypeScript. This solution is a practical example of how AWS CDK simplifies the development, deployment, and management of AWS resources in an efficient and "fashionable" way.

I'm thrilled to share with you how I've dabbled with the AWS CDK to build this serverless solution in just a few minutes. This project was an incredible opportunity to create a handy tool and gain understanding of CDK's workings.

Having had some experience with CloudFormation and Terraform, I found the shift to CDK intriguing. CDK allows you to define your cloud resources in the programming language you're comfortable with, making the entire development process more intuitive. Through this practical experience, we will explore the Amazon Polly service, which we'll dive into in the upcoming topics.

What's Covered?

What is AWS CDK?
What is Amazon Polly?
Text-to-Speech Application
Deploying with CDK
Final Thoughts and Further Exploration

What is AWS CDK?

Before diving into the details of our application, let me introduce you to the AWS CDK. AWS CDK is an open-source software development framework for defining cloud infrastructure as code (IaC) and provisioning it through AWS CloudFormation. It allows you to define your cloud resources using familiar programming languages, providing you with the advantages of the robustness, flexibility, and power of those languages.

The key concepts in CDK are Stacks and Constructs.

Stacks

A stack in AWS CDK is a unit of deployment. Each stack that you define with AWS CDK is transpiled into a CloudFormation stack and deployed as an atomic unit.

Constructs

Constructs, on the other hand, are the basic building blocks of AWS CDK apps. A construct represents a "cloud component" and encapsulates everything AWS CloudFormation needs to create the component.

By combining these concepts, you can break down your serverless application into manageable units, simplifying the process of building complex cloud resources.

What is Amazon Polly?

Now, let's talk about Amazon Polly, which is at the heart of our application. Amazon Polly is a service that turns text into lifelike speech. It uses advanced deep learning technologies to synthesize speech that sounds like a human voice.

In our Text-to-Speech function, we leverage Amazon Polly to read the text files and convert them into audio.

Amazon Polly pricing

For Amazon Polly’s Standard voices, the free tier includes 5 million characters per month for speech (~119 hours). For Neural voices, the free tier includes 1 million characters per month for speech(~23 hours)

The price can variyng aroud number of characters synthesized and the Model you use, here is the table of pricing examples from official Amazon Polly Pricing page:

Pricing Examples	Text Length	Speech Duration	Standard TTS Cost	Neural TTS Cost
1,000 requests, 1,000 characters per request	1 million characters	~23 hours, 8 min	$4.00	$16.00
10,000 requests, 100 characters per request	1 million characters	~23 hours, 8 min	$4.00	$16.00
2016 Amazon Shareholders Letter	1.3k characters, single page	~1 min. 40 sec	$0.005	$0.021
Average email message	~3.1k characters	~4 min	$0.01	$0.05
Typical news article	~6.5k characters, three pages	~9 min	$0.03	$0.10
"A Christmas Carol" by Charles Dickens	~165k characters, 64 pages	~3 hours 50 min	$0.66	$2.64
Storytelling with highlighted text for children: - Length of text for the story: 10k characters - Need for Speech Marks to synchronize highlighted text	10k characters of synthesized speech 10k characters of Speech Marks data	~13 min	$0.08	$0.32

Text to Speech pipeline

The interaction between the resources forms a serverless text-to-speech conversion pipeline. When a text file is added to the Text Bucket, it triggers a series of events that result in the creation of an audio file in the Audio Bucket and the updating of metadata in the DynamoDB table. Let's break it down:

S3 Buckets

Text Bucket: This bucket is used to store the text files that will be converted to speech. When a new text file is added to this bucket, it triggers a message to the Processing Queue.
Audio Bucket: This bucket is used to store the audio files that are the result of the text-to-speech conversion.

Both buckets have versioning disabled, use S3 managed encryption, and block all public access.

Free tier eligible:

Service	Limit	Description
Amazon S3	5 GB	Secure, durable, and scalable object storage infrastructure
Standard Storage	5 GB	-
Get Requests	20,000	-
Put Requests	2,000	-

DynamoDB Table

A DynamoDB table named MetadataTable is created to store metadata about the text and audio files as the status of conversion. It uses pay-per-request billing mode, which means you're only charged for the reads and writes that your application performs on the table.

Always free at this setup:

Service	Limit
Storage	25 GB per month
Provisioned write capacity units	25 per month
Provisioned read capacity units	25 per month

SQS Queue

An SQS queue named ProcessingQueue is created to manage the processing of the text files. When a new text file is added to the Text Bucket, a message is sent to this queue. The visibility timeout for the queue is set to 300 seconds, meaning that a message received from this queue will be invisible to other receiving components for 300 seconds.

Free at this level:

You can get started with Amazon SQS for free. All customers can make 1 million Amazon SQS requests for free each month. Some applications might be able to operate within this Free Tier limit.

Lambda Functions

Text to Speech Function: This function is triggered by messages in the SQS queue. It reads the text file from the Text Bucket, call the Amazon Polly SynthesizeSpeech API and saves the result audio mp3 to the audio bucket. It also updates the metadata in the DynamoDB table whith the conversion status and audio metadata. This function has read access to the Text Bucket, read and write access to the DynamoDB table, write access to the Audio Bucket, and permission to use Amazon Polly for speech synthesis.

Free tier eligible

1 million requests free
per month with the AWS Free Tier

To ensure we work within the limitations of the SynthesizeSpeech API, it is important to note the following restrictions:

The input text should not exceed 3000 billed characters (6000 total characters). It is worth mentioning that SSML tags are not counted as billed characters.
You can apply up to five lexicons to the input text.
The output audio stream, or synthesis, is limited to a maximum of 10 minutes. If the text exceeds this limit, any remaining speech will be cut off.

To overcome these limitations, an alternative option is to utilize the SpeechSynthesisTask API, which is asynchronous. This API allows Polly to directly place the audio into the designated s3 bucket. For more detailed information, please refer to the official documentation.

Using AWS CDK, we can model these resources as stacks and constructs, allowing us to build, replicate, and manage the infrastructure efficiently.

Deploying with CDK

Before you can deploy this project, you need to have the AWS CDK installed. If you haven't installed it yet, you can do so by running npm install -g aws-cdk. You also need to have your AWS credentials set up. You can follow the AWS CDK Getting Started guide for more information.

Once you have the AWS CDK installed and your credentials set up, you can deploy the project by following these steps:

Clone the repository:
git clone git@github.com:brenonaraujo/aws-cdk-serverless-text-to-speech.git
Navigate to the project directory:
cd aws-cdk-serverless-text-to-speech
Install the dependencies:
npm install
Compile the TypeScript code:
npm run build
Deploy the stack:
cdk deploy

Please note that the cdk deploy command will create real resources in your AWS account and may incur costs if you are out of the free tier period (12 months after account creation).

Once your are done, you can just destroy all the resources created:
cdk destroy

Testing our App

After the deployment process has finished, you can now go to your account and find the text bucket an put a text file in it. After a couple of seconds, you should see your audio file in the audio bucket.

Final Thoughts and Further Exploration

This journey with AWS CDK has been enlightening. It has enabled us to deconstruct complex applications into manageable components, thereby simplifying the creation, maintenance, and understanding of serverless solutions.

Amazon Polly has become another valuable asset. A few weeks ago, I was considering paying for a SaaS that offers exactly what we can achieve with Amazon Polly for 23 hours monthly without any upfront cost. It's almost like saying we can synthesize a book per month at a free tier (remember to respect copyright laws).

Thank you for accompanying me on this journey. Please feel free to share any feedback you may have.

Part II of this article includes a deep dive into the code.

Resources:
AWS CDK Documentation
AWS Lambda Documentation
Amazon S3 Documentation
Amazon DynamoDB Documentation
Amazon SQS Documentation
Amazon Polly Documentation