This project demonstrates building a containerized API management system for querying sports data. It leverages Amazon ECS (Fargate) for running containers, Amazon API Gateway for exposing REST endpoints, and an external Sports API for real-time sports data. The project showcases advanced cloud computing practices, including API management, container orchestration, and secure AWS integrations.
Project Architecture
Understanding the workflow
This workflow begins with a Python Flask backend app that fetches sports data from a SERP API endpoint.
The Python app is containerized using Docker for easy accessibility on Amazon ECR(Elastic Container Registry) as a publicly accessible docker image.
Amazon ECS Cluster uses Fargate to create and run containers, which allocate resources automatically for running containers.
Application Load Balancer handles distributing request evenly to the running containers on the ECS Cluster.
Amazon API Gateway add much more security by providing a public-facing URL endpoint for the API for client/user to request from. With API Gateway we can implement authorization, throttling which protects backend system from attacks.
Prerequisites
Register for a free account/subscription at serpapi.com and obtain your API Key
Install and configure AWS CLI to programmatically interact with AWS
Install Serpapi library in local environment “pip install google-search-results”
Install docker CLI and Docker Desktop to build and ush container images
Create AWS account with basic understanding of containers, APIs and Python
Technologies
- Cloud Provider: AWS
- Core Services: Amazon ECS (Fargate), API Gateway, CloudWatch
- Programming Language: Python 3.x
- Containerization: Docker
- IAM Security: Custom least privilege policies for ECS task execution and API Gateway
Project Structure
Check the Github repository here
Implementation
STEP 1: Docker Python Flask Application
The project folder contains a Dockerfile. This file defines how our application and dependencies should be packaged into a docker image.
docker build --platform linux/amd64 -t sports-api:1.0 .
STEP 2: Push Docker Image to ECR
- The first initial step is to create a repository on ECR to host the docker image.
aws ecr create-repository --repository-name <repo-name> --region <your-region>
- Pushing docker to ECR requires Authenticate to AWS
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin <AWS_ACCOUNT_ID>.dkr.ecr.us-east-1.amazonaws.com
- Tag Docker Image
docker tag sports-api:latest <AWS_ACCOUNT_ID>.dkr.ecr.us-east-1.amazonaws.com/nfl-sports-api:1.0
- Push Image to ECR
docker push <AWS_ACCOUNT_ID>.dkr.ecr.us-east-1.amazonaws.com/nfl-sports-api:1.0
Here's a screenshot of the AWS ECR repository
STEP 3: Set up ECS Cluster with Fargate to Deploy Image
- Create an ECS cluster
Fargate is a serverless architecture which need not provisioning of servers
- Go to the ECS Console → Clusters → Create Cluster
- Create a Task definition
Task definition tells ECS how to run the containers, including details like the container image to use, ports, and any environment variables needed by the container.
Choose the launch type (Fargate)
Add the container image from ECR.
Name the container
Set the container port to 8080.
Pass “sports_api_key” API key as an environment variable.
- Create a Service
- Specify the number of tasks you want to run (e.g., 2 or more tasks for high availability).
- A new security group (firewall rules) and set it to allow all TCP traffic from anywhere.
- Attach an Application Load Balancer (ALB) to the service, which routes user requests to the containers.
- Set up a health check for the ALB to ensure the application is functioning properly (/sports).
Go to the AWS loadbalancer console and click on the DNS endpoint to access the application
STEP 4: Integrate API Gateway
- Create a New REST API:
Go to the API Gateway Console.
Click on Create API and select REST API.
Name the API.
- Set Up Integration:
- Create a resource named “/sports”.
- Create a GET method for the resource.
- Choose HTTP Proxy as the integration type.
Enter the DNS name of the ALB that includes “/sports”
Deploy the API:
Deploy the API to a stage (e.g., “prod or dev”).
Note the endpoint URL provided by the API Gateway.
Key Lessons
While implementing the Sports API Management System, I learned several key lessons:
- Simplified Deployment with Docker: Docker containers made it easy to package and deploy the application by bundling everything needed to run the app. This ensured consistency across different environments.
- Serverless Infrastructure with AWS Fargate: Using AWS Fargate eliminated the need to manage underlying infrastructure, allowing me to focus on the application itself. Fargate handled the compute resources, making the deployment process seamless.
- Secure and Scalable API Exposure with API Gateway: Amazon API Gateway provided a secure and scalable way to expose the REST API to the internet. It added a layer of security and control, ensuring that only authorized users could access the API.
- High Availability and Load Balancing with ALB: The Application Load Balancer (ALB) distributed incoming traffic across multiple containers, ensuring high availability and improved performance. The ALB also performed health checks to ensure the application was functioning properly.
- Automated Scaling: With AWS tools like ECS, Fargate, and API Gateway, the system was ready to scale automatically based on demand. This reduced the overhead of managing infrastructure and allowed me to focus on the application.
- Cloud-Native Tools: Leveraging cloud-native tools simplified traditionally complex workflows, providing a solid foundation for containerized application deployment and API management. These learnings provided valuable insights into how cloud-native tools can simplify and enhance the deployment and management of containerized applications.
I hope you find this article and lab helpful, happy reading, happy building!!!
LinkedIn | Github | X
Top comments (0)