DEV Community

Gbenga Ojo-Samuel
Gbenga Ojo-Samuel

Posted on

Streamlining NCAA Game Highlights with RapidAPI, Docker, and AWS MediaConvert

In this project, we demonstrate how to leverage RapidAPI, Docker, and AWS MediaConvert to create an efficient pipeline for obtaining and processing NCAA game highlights. Additionally, we use Terraform to deploy an EC2 instance that automatically establishes an SSH connection and runs Python scripts to manage the workflow seamlessly. This setup simplifies the process of acquiring high-quality sports media content and converting it into usable formats for distribution or analysis.

Prerequisites
Before you get started, ensure you have the following:

  1. RapidAPI Account: Sign up for a RapidAPI account and subscribe to an API that provides NCAA game highlights.
  2. AWS Account: Set up Role with access to AWS MediaConvert and an S3 bucket.
  3. Docker Installed: Install Docker on your system to run the containerized workflow. We will install Docker on the EC2 instance
  4. Terraform Installed: Ensure Terraform is installed for infrastructure deployment.
  5. Basic CLI Knowledge: Familiarity with using command-line tools for API requests, AWS configurations, and Terraform commands.

Technical Diagram

Image description

RapidAPI is a robust API marketplace offering access to a variety of data sources, including NCAA game highlights. By integrating RapidAPI into our workflow, we can easily retrieve game highlight videos and relevant metadata.

Docker containers ensure the workflow is portable and reproducible across environments. By encapsulating the necessary dependencies and scripts in a Docker image.

AWS MediaConvert is a powerful tool for converting media files into formats optimized for various use cases, such as streaming or archival. In this project, MediaConvert processes the raw video fetched from RapidAPI and converts it into desired output formats.

Deploying an EC2 Instance with Terraform
Terraform automates the deployment of an EC2 instance that serves as the base for running the Python scripts and Dockerized workflow. This step involves:

Terraform Configuration:

  • Create a Terraform script that includes the following:
  • An EC2 instance with appropriate IAM roles and security groups to enable necessary access.
  • Configuration using the null_resource provisioner to upload the required Python scripts to the EC2 instance and execute them automatically.

Initialize Terraform
Clone the git repo and cd to the project directory

https://github.com/OjoOluwagbenga700/NCAAGameHighlight.git

Run terraform init to initialize the Terraform working directory.

Image description

Plan the Deployment

Use terraform plan to preview the resources that Terraform will create.

Image description

Apply the Configuration

Execute terraform apply –auto-approve to deploy the infrastructure.

Automated Script Execution

The EC2 instance is configured using the null_resource provisioner in Terraform to both upload the necessary Python scripts and execute them automatically.

The remote-exec provisioner ensures that the EC2 instance is prepared by installing Docker, setting up the environment, and running the highlight-fetching workflow in a containerized environment.

Image description

*Verify Deployment on AWS *

Image description

Image description

Image description

Image description

Conclusion

Congratulations! We have successfully demonstrated how to fetch JSON data from RapidAPI, transform it into a video, and enhance its quality using AWS MediaConvert.

This project underscores the power of integrating APIs, Docker containers, and AWS cloud tools to build a streamlined and automated workflow. By leveraging Terraform for infrastructure as code, we ensured consistent deployment of an EC2 instance to automate the entire process, from fetching NCAA game highlights to enhancing them for better quality.

Top comments (0)