NOTE: This is a repost of my Original Blog
🎬 Lights, camera, action! You've captured some fantastic videos, but there's a tiny twist to our story. We're not making the videos silent film stars; we're giving the soundtrack the limelight it deserves.
🌟 In this whimsical journey through the cloud, I'll show you how to take your video uploads and, with the help of AWS S3, Lambda, and AWS MediaConvert, conjure the audio right out of them. It's like pulling a rabbit out of a hat, but in our world, I'm pulling melodies out of videos! Join me for a magical adventure where your videos reveal their musical talents, and I promise you a touch of humour, a pinch of technical enchantment, and a whole lot of AWS magic. 🪄🎶
So no more waiting, Let's dive in !!
Introduction
In this blog, our main focus will be on crafting a workflow within the AWS platform, enabling users to effortlessly extract audio (MP4) from their video files (MP4). To achieve this, we'll harness the power of several key AWS services:
Amazon S3
AWS Lambda
AWS Media Convert
AWS IAM
Functional Flow
As you can see in the above workflow diagram, It explains how the video file flows and you get the desired MP4 Audio File. Below are the high-level steps of the workflow:
The user uploads the MP4 video file to an S3 Bucket, Let's name it here as
video2audio-source-video-bucketS3 Event Notification is triggered on file upload event on the bucket
video2audio-source-video-bucket. Event Notification hits the desired target i.e., Lambda function (named video2AudioLambda) in this case.The Lambda function creates a Media Convert Job with the respective Media Convert Properties.
Media Convert Job fetches the Video file from the S3 bucket from which Audio has to be extracted
Media Convert Job sends the generated audio file to the output S3 bucket that is configured in the Media Convert Properties. Let's consider the output bucket name here as
video2audio-target-audio-bucket.The audio file is available for download from the S3 bucket.
Building the system
Creating the S3 Buckets
Login to the AWS Console
Go to the S3 Service Dashboard and click on "Create bucket"
You will land on a page where you have to fill in the name of the S3 bucket as per your choice (here I have used
video2audio-source-audio-bucketfor bucket to be used for MP4 video files andvideo2audio-target-video-bucketfor the bucket to be used to export the MP4 Audio).
Important Note: S3 bucket names are to be kept globally unique. In case the name is not available then you can use another name accordingly
Choose the region according to your choice leaving the rest of the details as it is and click on "Create Button"
Follow the same steps for both of the source and target buckets and the buckets will be created which you can find on the S3 dashboard.
Setting up the Lambda function
In our case, the Lambda function will be triggered by the S3 event notification on the upload of a file to the source S3 bucket.
Lambda function will make use of "Create Job" API call to AWS Media Convert service. Where Media Convert will be responsible for the Video to Audio Conversion.
So as the Lambda function has to access the Media Convert service to create a Job, We will have to create a role which has permissions to create the job. Let's create one.
Creating IAM Role for Lambda Function
Jump on to the IAM Service and them in the Roles section for creating the role.
Click on Create Role.
-
You will land on the IAM Role creation page which shows 3 steps:
- Selecting Trusted Entity: We are creating this role to be attached to a AWS Service i.e., Lambda here. So we will select "AWS Lambda" in trusted Entity Types. For the use case, we will select "Lambda".
- Add Permissions: We will add the AWS MAnaged Policies to our role named "AWSElementalMediaConvertFullAccess" which will allow our Lambda to perform respective operations.
Name, Review & Create: In the Final stage you can add the name for your role (here video2AudioMediaConvertRole). Review the permissions and create the role.
- So now as our Lambda function Role is created, we will jump back to understand further the requirements of Media Convert Properties.
Understanding the Media Convert "Create Job" API Call configuration
In the Create Job call, we need to send the configuration properties of the Media Convert Job. Configuration details for the job will be in JSON format.
-
Below Configuration Properties are to be set in the JSON:
-
Queue ARN: The "Queue" is set to a specific AWS Elemental MediaConvert queue. This is where the job will be processed. The "Default" queue is commonly used when you don't have specific queue requirements. We will be using the Default Queue for which ARN is of format:
arn:aws:mediaconvert:<AWS-Region>:<AWS-Account-ID>:queues/DefaultFor this blog post, I have used "us-east-1" region. - Role: The "Role" specifies the IAM role that Media Convert assumes to access the necessary resources.
- OutputGroups: This section defines the output settings. In our case, it's set to create a single "File Group" with an MP4 container, indicating that the output will be in MP4 format. Inside the "File Group," it defines the audio settings. The audio source is selected as "Audio Selector 1," and it uses AAC codec settings with a bitrate of 96,000 bits per second and a sample rate of 48,000 Hz.
- OutputGroupSettings: This section specifies the settings for the output file. It defines the destination S3 bucket where the output audio file will be saved.
- Inputs: Here, it defines the input settings, including the source video file path in our S3 bucket.
- Below JSON properties are to be used in our case.
-
Queue ARN: The "Queue" is set to a specific AWS Elemental MediaConvert queue. This is where the job will be processed. The "Default" queue is commonly used when you don't have specific queue requirements. We will be using the Default Queue for which ARN is of format:
{
"Queue": "arn:aws:mediaconvert:us-east-1:<AWS:ACCOUNT:ID>:queues/Default",
"UserMetadata": {},
"Role": "arn:aws:iam::<<AWS:ACCOUNT:ID>:role/mediaConvertRole",
"Settings": {
"OutputGroups": [
{
"Name": "File Group",
"Outputs": [
{
"ContainerSettings": {
"Container": "MP4",
"Mp4Settings": {},
},
"AudioDescriptions": [
{
"AudioSourceName": "Audio Selector 1",
"CodecSettings": {
"Codec": "AAC",
"AacSettings": {
"Bitrate": 96000,
"CodingMode": "CODING_MODE_2_0",
"SampleRate": 48000,
},
},
}
],
}
],
"OutputGroupSettings": {
"Type": "FILE_GROUP_SETTINGS",
"FileGroupSettings": {
"Destination": "s3://"
+ output_bucket
+ "/"
+ output_file_key
},
},
}
],
"Inputs": [
{
"AudioSelectors": {
"Audio Selector 1": {"DefaultSelection": "DEFAULT"}
},
"VideoSelector": {},
"TimecodeSource": "ZEROBASED",
"FileInput": "s3://" + input_bucket + "/" + input_file_key,
}
],
},
}
NOTE: Make sure to replace the Bucket Name, Role Name, Account ID, Region and File Name according to your usage.
Wait now you might be thinking about where this Role in the above property came from. So, as you remember, Our Media Convert Job will pull the changes from the source S3 bucket, Convert the Video to Audio and Push the Audio file to S3.
So to perform these operations, the Media Convert Job should be assigned a role with respective permissions. Let's understand the Role creation for the Media Convert Job in our case below.
Creating IAM Role for Media Convert Job
Jump on to the IAM Service and then in the Roles section to create the role.
Click on Create Role.
-
You will land on the IAM Role creation page which shows 3 steps:
- Selecting Trusted Entity: We are creating this role to be attached to a AWS Service i.e., Media Convert here. So we will select "MediaConvert" in trusted Entity Types. For the use case, we will select "MediaConvert".
- Add Permissions: For Media Convert case, we will see that, we already have two AWS MAnaged Policies attached i.e., AmazonS3FullAccess & AmazonAPIGatewayInvokeFullAccess. These two policies are enough for our case as we just need access to the S3 bucket via Media Convert.
- Name, Review & Create: In the Final stage you can add the name for your role (here mediaConvertRole). Review the permissions and create the role.
So, now that we are done with the Role creation tasks, We can move on to the task of creating our Lambda Function using Python.
Creating the Lambda Function
Go to the AWS Lambda Service in AWS Console
Click on "Create a function" button. You will land on the Lambda creation page.
-
Select the following options:
- Author from Scratch as we are going to write our lambda function code
- Name as video2AudioLambda
- Select Python 3.11 as the lambda runtime (You can make use of any other runtime too but the code changes will be needed accordingly)
- Expand "Change default execution role" Select "Use an existing Role" & then choose the IAM role that we created for Lambda (video2AudioMediaConvertRole).
- Click on "Create Function" and your raw Lambda function will be created.
Wait guys, the job is not yet done, the actual thrill is below !!!
We have to make use of the Python code given below. Paste it down in our Lambda console and Click on Deploy.
So this lambda function written in Python creates a
mediaconvert clientby making use ofboto3library. Then this client is used for creating a media conversion job using "Create Job" API call.IMPORTANT: There is no need to hardcode the name of the Source S3 bucket here because, later on, we are going to configure the S3 Event Notification, which is then going to trigger the Lambda function. So the Event object will have all the details of the Bucket and Object that was uploaded to S3.
However, we need to hardcode the target bucket name & the target audio file name (Update the bucket name as per the name that you have used).
We have made use of the Media Convert properties that were explained in the above section and have passed the same to the
create_jobfunction call
AWS Lambda Python Code
import boto3
import os
def lambda_handler(event, context):
# Initialize the MediaConvert client
# Get the Endpoint URL from --> MediaConvert --> Account --> API Endpoint
mediaconvert = boto3.client(
"mediaconvert",
endpoint_url="https://vasjpylpa.mediaconvert.us-east-1.amazonaws.com",
)
input_bucket = str(event["Records"][0]["s3"]["bucket"]["name"])
input_file_key = str(event["Records"][0]["s3"]["object"]["key"])
output_bucket = "video2audio-target-audio-bucket"
output_file_key = "output-audio"
# Create a job settings JSON
job_settings = {
"Queue": "arn:aws:mediaconvert:us-east-1:666541739853:queues/Default",
"UserMetadata": {},
"Role": "arn:aws:iam::666541739853:role/mediaConvertRole",
"Settings": {
"OutputGroups": [
{
"Name": "File Group",
"Outputs": [
{
"ContainerSettings": {
"Container": "MP4",
"Mp4Settings": {},
},
"AudioDescriptions": [
{
"AudioSourceName": "Audio Selector 1",
"CodecSettings": {
"Codec": "AAC",
"AacSettings": {
"Bitrate": 96000,
"CodingMode": "CODING_MODE_2_0",
"SampleRate": 48000,
},
},
}
],
}
],
"OutputGroupSettings": {
"Type": "FILE_GROUP_SETTINGS",
"FileGroupSettings": {
"Destination": "s3://"
+ output_bucket
+ "/"
+ output_file_key
},
},
}
],
"Inputs": [
{
"AudioSelectors": {
"Audio Selector 1": {"DefaultSelection": "DEFAULT"}
},
"VideoSelector": {},
"TimecodeSource": "ZEROBASED",
"FileInput": "s3://" + input_bucket + "/" + input_file_key,
}
],
},
}
# Start the MediaConvert job
response = mediaconvert.create_job(**job_settings)
print(str(response))
return {"statusCode": 200, "body": "MediaConvert job started successfully."}
You have to copy the above code and paste it to the Lambda console and then click on Deploy!!
- In above Python code you might have noticed that we have made use of an endpoint URL (
https://vasjpylpa.mediaconvert.us-east-1.amazonaws.com)to create the mediaConvert Client. You can get that endpoint URL by going into your Media Convert Service in AWS Console at path:MediaConvert --> Account --> API Endpoint
Configuring S3 Event Notification
Go to AWS S3 Console
Click on your bucket (
video2audio-source-video-bucket). Then go to "Properties" section of the same.On Scrolling down, you will find the section for "Event Notification". You have to click on "Create Event Notification"
You will land on a page to fill in the details for S3 Event trigger action and the target.
You can name your event and then select the format of the file that we want to trigger our Audio Conversion action on. So here we are using ".mp4" as the source file format.
In the Event Types, Just check on the "All Object Creation Events", as we have to trigger this conversion on every file creation/updation event
Scroll down to the bottom to fill in the Destination details !! In our case it is a lambda function named "video2AudioLambda" (You can also choose to fill the ARN of the lambda function)
And then Just click on the save changes.
Woohoooo we have finally completed the Build phase !!! Now it is the time to test our design....
Testing the setup
We have to upload a MP4 video file in the source bucket (here
video2audio-source-video-bucket)Then in a few seconds, you can check your target bucket (here
video2audio-target-audio-bucket) And you will finally see an MP4 Audio file over there.
Advanced Learning Capsule points
- Checking Lambda Executions: In case you want to check the Lambda Execution, you will have to first Assign an IAM policy to your Lambda Role for CloudWatch Logs Access. Which will then allow your Lambda Function to publish logs to your CloudWatch Log Dashboard.
You can access the logs via Going into the Log Group and checking for the LogGroup
"/aws/lambda/<your-lambda-function-name>"
-
Checking Job Submissions in Media Convert Service:
- Go to the Media Convert Service on AWS Console.
- Click on the Jobs option in the Menu on the Left.
- You will find all of the Job Submissions and you can check the status of each respective job easily.
As we conclude this journey into the fascinating realm of audio extraction from video, I hope you've not only gained valuable insights but also a newfound appreciation for the versatility of AWS. The possibilities are endless when you combine technology with creativity.
Thank you for being a part of this adventure. Until next time, happy cloud computing!!
If you have any questions, ideas, or experiences you'd like to share, I'm all ears. Let's keep the conversation going in the comments section below. And remember, this is just the beginning; there's a world of innovation waiting for you in the cloud. Stay curious, keep exploring, and keep making technology work for you. Follow me on social media, where I promise to share cloud wisdom with a side of chuckles:
LinkedIn: Connect with me on LinkedIn, where my cloud prowess is only rivalled by my talent for finding the perfect GIF for every situation. 🚀💼 hardeepjethwani@LinkedIn
TopMate: Looking for a fellow cloud aficionado to share a virtual coffee with or brainstorm your next AWS masterpiece? Find me on TopMate! Because let's face it, cloud enthusiasts need to stick together. ☕🤝 hardeepjethwani@topmate
Instagram: For behind-the-scenes glimpses of my cloud adventures and occasional 'AWS Gone Wild' stories that even AWS engineers find amusing. 📸🌩️ hardeepjethwani@Instagram
X: Join the cloud conversation on Twitter, where I drop cloud knowledge and quirky cloud memes faster than you can say 'Elastic Beanstalk.' 🐦☁️ hardeepjethwani@X
So, whether you're seeking cloud advice, a good laugh, or simply a friendly chat about cloud and coffee preferences, I'm just a click away on these cloud-tastic platforms. See you in the cloudisphere, fellow cloud builders!" 🌍☁️😄
Want to support my cloud adventures and keep the coffee flowing? Feel free to buy me a virtual coffee. After all, coffee is the secret sauce behind every successful cloud deployment. ☕🙌
Top comments (0)