“Challenges faced to find the solution of how to convert the video file format from .mp3 to .ts format within some seconds”. I have checked different ways so that I can get my video file format conversion within a few seconds and within my budget. I got the solution by using an amazon elastic transcoder service in which I am able to set the configuration of the transcoding pipeline and able to create my custom presets as well. The cost of the pipeline is in my budget. Even I can create different presets as 1M, 1.5M, 600K and many more as per my need or can use the existing preset as well. I have created code for lambda function with s3 trigger which automatically generates the output in s3 bucket whenever i do input in my bucket of any video file format. My output will be based on the lambda code of preset value which has been set.
Amazon Elastic Transcoder is media transcoding in the cloud. It is designed to be a highly scalable, easy to use and a cost effective way for developers and businesses to convert (or “transcode”) media files from their source format into versions that will playback on devices like smartphones, tablets and PCs.
What's New with Amazon Elastic Transcoder??
- Amazon Elastic Transcoder Adds Support For Clip Stitching
- Amazon Elastic Transcoder Available In The Asia Pacific (Mumbai) Region
- Amazon Elastic Transcoder Now Publishes Operational Metrics to Amazon CloudWatch
- Amazon Elastic Transcoder Available In The Asia Pacific (Sydney) Region
- Amazon Elastic Transcoder Adds Support For WAV
- Amazon Elastic Transcoder Now Supports MPEG-DASH
- Amazon Elastic Transcoder Adds Support for VP9
To learn more, read the AWS Elastic Transcoder.
AWS Lambda is a serverless compute service that lets you run code without provisioning or managing servers, creating workload-aware cluster scaling logic, maintaining event integrations, or managing runtimes. With Lambda, you can run code for virtually any type of application or backend service - all with zero administration. Just upload your code as a ZIP file or container image, and Lambda automatically and precisely allocates compute execution power and runs your code based on the incoming request or event, for any scale of traffic. You can set up your code to automatically trigger from 140 AWS services or call it directly from any web or mobile app. You can write Lambda functions in your favorite language (Node.js, Python, Go, Java, and more) and use both serverless and container tools, such as AWS SAM or Docker CLI, to build, test, and deploy your functions. To learn more, read the AWS Lambda.
In this post, you will get to know how to convert any input video file format in S3 bucket to any other video file format using Elastic Transcoder and Lambda. Here I have taken two S3 buckets and then created a transcode pipeline with preset. And set a lambda code with IAM role and set a S3 trigger on it which will automatically generate the output video files of different formats in the s3 bucket.
Prerequisites
You’ll need an Amazon Simple Storage Service for this post. Getting started with amazon Simple Storage Service provides instructions on how to create a bucket in simple storage service. For this blog, I assume that I have two buckets for input and output of videos.
Architecture Overview
Diagram
The architectural diagrams show the overall deployment architecture with data flow, AWS S3, AWS Elastic Transcoder and AWS Lambda.
Solution overview
The blog post consists of the following phases:
- Create AWS Elastic Transcoder Pipeline and Presets
- Create IAM role and Lambda function with S3 trigger
- Testing of video file format and check of cloudwatch logs
I have two S3 bucket for input and output of video as below →
Phase 1: Create AWS Elastic Transcoder Pipeline and Presets
- Open the AWS Elastic Transcoder console and create a new pipeline.
- In the new pipeline, give the pipeline name as transcodepipeline. Choose the source bucket and IAM role to be as default only. In configuration of s3 bucket for transcoded files and playlists, give bucket name as destination bucket and storage class as standard.
- Click on add permission and choose grantee type as Amazon S3 Group. And Choose grantee as All Users with Open/Download Access only. And leave other options as default and then create the pipeline.
- We can even schedule the job for our pipeline as per requirement from Jobs option.
- We can create a new preset or use the existing system preset as well. Here I have created 4 presets of the same configuration for different Generic as HLS 400k, HLS 600k, HLS 1.5M and HLS 2M.
- Create a new preset named as Preset: HLS 400k. Choose container type as ts and tick all available settings. Set video max frame rate as 30 and leave other settings as default. Checkout other configurations as well for HLS 600k, HLS 1.5M and HLS 2M.
Phase 2: Create IAM role and Lambda function with S3 trigger
- In AWS IAM console, create a role named as LambdaS3CW with policy as elastic transcoder full access, s3 full access and cloudwatch full access.
- In AWS Lambda console, Create a lambda function named as transcoderfunction with IAM role LambdaS3CW and Node.js version. In the code section, set the pipeline id from the pipeline created and preset id from preset created. Also set the preset, thumbnail and playlist as per requirement in the code.
//Use the code snippet
'use strict';
var AWS = require('aws-sdk');
var s3 = new AWS.S3({
apiVersion: '2012–09–25'
});
var eltr = new AWS.ElasticTranscoder({
apiVersion: '2012–09–25',
region: 'ap-south-1'
});
exports.handler = function(event, context) {
console.log('Executing Elastic Transcoder');
var bucket = event.Records[0].s3.bucket.name;
var key = event.Records[0].s3.object.key;
var pipelineId = '1618142748073-skxu76';
if (bucket !== 'transcodevideo-src') {
context.fail('Incorrect Input Bucket');
return;
}
var srcKey = decodeURIComponent(event.Records[0].s3.object.key.replace(/+/g, " ")); //the object may have spaces
var newKey = key.split('.')[0];
var fk= "400K";
var sk= "600K";
var om= "1.5M";
var tm= "2M";
var th= "Thumb"
var params = {
PipelineId: pipelineId,
OutputKeyPrefix: newKey + '/',
Input: {
Key: srcKey,
FrameRate: 'auto',
Resolution: 'auto',
AspectRatio: 'auto',
Interlaced: 'auto',
Container: 'auto'
},
Outputs: [{
Key: fk + '/' +'hls4K-' + newKey + '.ts',
ThumbnailPattern: th +'/'+'thumbs4K-{count}',
PresetId: '1562047543931-59morw',//HLS v3 400K/s
SegmentDuration:"5"
},
{
Key: sk + '/' +'hls6K-' + newKey + '.ts',
ThumbnailPattern: th +'/'+'thumbs6K-{count}',
PresetId: '1562047718852-9h1hs7', //HLS v3 600K/s
SegmentDuration:"5"
},
{
Key: om + '/' +'hls1.5M-' + newKey + '.ts',
ThumbnailPattern: th +'/'+'thumbs1.5M-{count}',
PresetId: '1562047826616-c3wgvg', //HLS v3 1.5M/s
SegmentDuration:"5"
},
,
{
Key: tm + '/' +'hls2M-' + newKey + '.ts',
ThumbnailPattern: th +'/'+'thumbs2M-{count}',
PresetId: '1562047886667-378v0i', //HLS v3 2M/s
SegmentDuration:"5"
}
],
Playlists: [{
Format: 'HLSv3',
Name: 'hls-'+ newKey,
OutputKeys: [ fk+ '/' +'hls4K-'+ newKey + '.ts',sk + '/' +'hls6K-' + newKey + '.ts',om + '/'
+'hls1.5M-' + newKey + '.ts',,tm + '/' +'hls2M-' + newKey + '.ts']
}]
};
console.log('Starting Job');
eltr.createJob(params, function(err, data){
if (err){
console.log(err);
} else {
console.log(data);
}
context.succeed('Job well done');
});
};
3.Set the lambda function timeout as 5 min 0 sec and set a S3 trigger for the source bucket. So that whenever there is an input of any video file format in the source bucket the trigger automatically converts the video format as per preset and saves the output in the destination bucket.
4.Deploy and test the lambda code to check the error in function.
Phase 3: Testing of video file format and check of cloudwatch logs
- I have input the .mp4 video in the source bucket and got the results of the video in a different file format in the destination bucket.
- In the cloudwatch log group, we can see the logs of the video transcoding activity. And can also monitor the success or error of lambda function(or code) activity in lambda.
Clean-up
Delete the environment as: S3 bucket, elastic transcoder(preset, pipeline), IAM role, Lambda function and cloudwatch log group.
Pricing
I review the pricing and estimated cost of this example.
For Elastic Transcoder →
Cost of SD content transcoded for 20 minutes = $0.30
For Lambda →
Cost of request and lambda-GB per second = $0.00
For Simple Storage Service →
Cost of APS3-Requests = $0.01
For cloudwatch, data transfer and simple notification (under free tier) →
Cost = $0.00
Total Cost = $(0.30+0.00+0.01+0.00) = $0.31
Summary
In this post, I have shown you how to convert any input video file format in S3 bucket to any other video file format using Elastic Transcoder and Lambda.
For more details on elastic transcoder and lambda service, Checkout Get started with AWS Elastic Transcoder and AWS Lambda, open the AWS Elastic Transcoder console and AWS Lambda console. To learn more, read the AWS Elastic Transcoder documentation and AWS Lambda documentation.
Thanks for reading!
Connect with me: Linkedin
Top comments (1)
Thanks for the article. Very helpful for me.
I have 100GB mov video on s3 needs to be converted to mp4. As per my knowledge lamda have limitation of 15 minutes of time.
Is there any way to overcome this or any other service.
Thank You.