NOTE: Do not split in two parts, there isn't enough text here to justify splitting and the article focuses on the solution not discussing the choices.
Need to generate video thumbnails efficiently and cost-effectively at scale? Let's build a truly serverless solution using AWS Lambda that costs just pennies to run, compared to using dedicated media processing services.
What we are going to build
The solution consists of a Node.js Lambda function that:
- Processes common video formats
- Scales based on workload
- Implements retry logic for failed operations
- Deploys via Infrastructure as Code
- Costs fraction of a cent per video to run
Why Custom
It's not super easy, or cheap, to generate thumbnails at scale. The cost factor is especially important in case of videos - with images all you have to do is resize, crop and store the output of the same type. You can offload this responsibility to third-party cloud services to focus on delivering other features, or with just a little bit of work perform the task without leaving your AWS VPC. With videos though the case is different. Video files are much larger, we have to support plenty of different encoding standards, and the end result is no longer a video - we are essentially extracting still images.
AWS Native = Super Expensive
When researching options I always turn to solutions native to the platform the application is on. In AWS that's MediaConvert or MediaLive. Both are great when you need professional-grade video processing, but when all you want is to grab a thumbnail from a video... well, they sure can do it but are they designed to handle such use case? Not really.
As surprising as it may be, AWS does not have a service dedicated to generating thumbnails. Available solutions focus on other use-cases such as providing support for streaming media or running advanced video transformation tasks.
The problems are quite obvious when you look at the requirements for building such feature with these services
- when working with AWS Media services it's not possible to create a processing pipeline that does not have a video output defined - you are required to process a whole video and discard the result only to use the thumbnails that are a byproduct of that process
- as such it's super expensive as a thumbnail generator - paying $0.0075 per minute of processed video may not feel like much but it's adding up really quick - for 1,000 videos, each 15 minutes long the cost of processing would be over $100
Should generating a few video thumbnails cost more than your morning coffee? ☕ It's simply because as powerful as those services are, it's and overkill for simple tasks like thumbnail generation.
The real cost of AWS Media services isn't just in dollars - it's in the complexity you often don't need.
Each time I come across a new requirement my mind tunes itself into the "finding the perfect tool for the job" mode. I've been preaching the importance of not going with what you know and always exploring as many alternatives as possible that I may start sounding like a broken record... but I guess I like the tune that record is playing! 💽🎶
You can also call it a medical condition. I am fully aware of my engineering OCD issues... 😅
But I digress...
Beyond AWS
Sure, there is other solutions out there, but they often come with their own headaches:
- External services usually charge per API call or amount of data processed
- You must upload your videos to external services for processing which means even more cost for egress
- They may not scale well, leaving you with handling throttling
A Custom Purpose-Built Solution
Let's build something that's not just cheaper, but also laser-focused on what we actually need - a serverless solution that generates video thumbnails for literal pennies. 🪙
The system uses these AWS services and tools:
- Amazon S3 - Storage for source videos and generated thumbnails
- AWS Lambda - Serverless compute environment
- FFmpeg - Video processing framework
- Docker - Container packaging for FFmpeg and Lambda code
- Amazon SQS - Message queue for processing coordination
When a video is uploaded to the source S3 bucket, it triggers an event that queues the processing request. A Lambda function picks it up and processes it using FFmpeg running in a Docker container. The generated thumbnails are then stored in a target S3 bucket. Recoverable transient issues such as throttling or infrastructure-related problems are automatically re-tried, while all other failed events are automatically sent to a dead-letter queue for auditing purposes.
The service automatically generates video thumbnails in two sizes. The larger version includes a semi-transparent video icon in the center of the frame, helping users quickly identify video content.
Sample project
- Pull from GH
Service
- orchestration Video Processing Util
- FFmpeg
- two thumbnail types
- different sizes one with an overlay FFmpeg in a Container
- this is how we make Ffmpeg CLI available for Lambda
- Dockerfile Deployment
- Container build
- Serverless deployment
- Dockerized Lambda definition Testing
- int
- e2e Serverless
- anything else at this point?
The Magical Container 🐳
Here's our Dockerfile that packages FFmpeg with Lambda:
Show Me the Money! 💰
Let's break down the costs for processing 1000 videos per month:
AWS MediaConvert
- $0.08 per minute of video
- 1000 videos × $0.08 = $80
Our Solution
- Lambda: 1024MB × 10s × 1000 = $0.17
- S3: Storage + GET/PUT = $0.05
- Total: $0.22
That's a 99.7% cost reduction! 🎉
What Could Go Wrong? 🤔
While this solution is awesome, it's not without its gotchas:
- Memory Usage: FFmpeg can be memory-hungry. If you're processing 4K videos, you might need to bump up the Lambda memory.
- Timeout Limits: For very long videos, you might hit Lambda's timeout. Consider using step functions for those cases.
- Cold Starts: The container is quite large, so first invocations might be slower.
What's Next? 🎬
This is just the beginning! You could extend this solution to:
- Generate multiple thumbnail sizes
- Extract video metadata
- Create preview GIFs
- Add video watermarks
Wrapping Up 🎁
We've built a cost-effective, scalable solution for video thumbnail generation that won't break the bank. No more paying for features you don't need!
Remember: Sometimes the best solution isn't the most expensive or complex one - it's the one that does exactly what you need, nothing more, nothing less.
Note
Found this helpful? Consider following me for more AWS and serverless content! And if your thumbnails come out looking like modern art instead of your video... well, check your video format first, then drop a comment below! 😄
All jokes aside, I'd love to hear about your experiences with video processing in AWS. Have you found other creative ways to optimize costs? Share in the comments!
Disclaimer
While this solution has been battle-tested in production, please test thoroughly in your own environment before deploying. If anything catches fire, I have a great recipe for marshmallows! 🔥
--- My notes - talking points for the article
Why docker with Lambda - not the first choice, sometimes the only choice, layers alternative
It's super cheap to run compare with AWS Media services
It's fast, run comparison on different file sizes
Testable Ffmpeg
Top comments (0)