In this project, we demonstrate how to leverage RapidAPI, Docker, and AWS MediaConvert to create an efficient pipeline for obtaining and processing NCAA game highlights. Additionally, we use Terraform to deploy an EC2 instance that automatically establishes an SSH connection and runs Python scripts to manage the workflow seamlessly. This setup simplifies the process of acquiring high-quality sports media content and converting it into usable formats for distribution or analysis.
Prerequisites
Before you get started, ensure you have the following:
- RapidAPI Account: Sign up for a RapidAPI account and subscribe to an API that provides NCAA game highlights.
- AWS Account: Set up Role with access to AWS MediaConvert and an S3 bucket.
- Docker Installed: Install Docker on your system to run the containerized workflow. We will install Docker on the EC2 instance
- Terraform Installed: Ensure Terraform is installed for infrastructure deployment.
- Basic CLI Knowledge: Familiarity with using command-line tools for API requests, AWS configurations, and Terraform commands.
Technical Diagram
RapidAPI is a robust API marketplace offering access to a variety of data sources, including NCAA game highlights. By integrating RapidAPI into our workflow, we can easily retrieve game highlight videos and relevant metadata.
Docker containers ensure the workflow is portable and reproducible across environments. By encapsulating the necessary dependencies and scripts in a Docker image.
AWS MediaConvert is a powerful tool for converting media files into formats optimized for various use cases, such as streaming or archival. In this project, MediaConvert processes the raw video fetched from RapidAPI and converts it into desired output formats.
Deploying an EC2 Instance with Terraform
Terraform automates the deployment of an EC2 instance that serves as the base for running the Python scripts and Dockerized workflow. This step involves:
Terraform Configuration:
- Create a Terraform script that includes the following:
- An EC2 instance with appropriate IAM roles and security groups to enable necessary access.
- Configuration using the null_resource provisioner to upload the required Python scripts to the EC2 instance and execute them automatically.
Initialize Terraform
Clone the git repo and cd to the project directory
https://github.com/OjoOluwagbenga700/NCAAGameHighlight.git
Run terraform init
to initialize the Terraform working directory.
Plan the Deployment
Use terraform plan
to preview the resources that Terraform will create.
Apply the Configuration
Execute terraform apply βauto-approve
to deploy the infrastructure.
Automated Script Execution
The EC2 instance is configured using the null_resource provisioner
in Terraform to both upload the necessary Python scripts and execute them automatically.
The remote-exec provisioner
ensures that the EC2 instance is prepared by installing Docker, setting up the environment, and running the highlight-fetching workflow in a containerized environment.
*Verify Deployment on AWS *
Conclusion
Congratulations! We have successfully demonstrated how to fetch JSON data from RapidAPI, transform it into a video, and enhance its quality using AWS MediaConvert.
This project underscores the power of integrating APIs, Docker containers, and AWS cloud tools to build a streamlined and automated workflow. By leveraging Terraform for infrastructure as code, we ensured consistent deployment of an EC2 instance to automate the entire process, from fetching NCAA game highlights to enhancing them for better quality.
Top comments (0)