Nonso Echendu

Posted on Feb 13

Building a FastAPI Book API with CI/CD Pipelines (Using Github Actions) and Docker Deployment

In this article, I’ll walk you through the development of a FastAPI-based book API, complete with a Continuous Integration (CI) and Continuous Deployment (CD) pipeline. This project was part of a Devops challenge with HNG, and is aimed to retrieve book details by ID, and deployed to an AWS ec2 instance using Docker.

Project Overview

The repository provided already included a predefined structure, and we had to ensure that all endpoints were correctly implemented and accessible via Nginx.

These were the goals to be achieved in this challenge:

Add an endpoint to retrieve a book by its ID, so that it is accessible via this url path /api/v1/books/{book_id}.
Set up a CI pipeline that automates tests whenever a pull request is made to the main branch.
Set up a CD pipeline that automates the deployment process whenever changes are pushed to the main branch.
Serve the application over NGINX

Project Structure

fastapi-book-project/
├── api/
│   ├── db/
│   │   ├── __init__.py
│   │   └── schemas.py      # Data models and in-memory database
│   ├── routes/
│   │   ├── __init__.py
│   │   └── books.py        # Book route handlers
│   └── router.py           # API router configuration
├── core/
│   ├── __init__.py
│   └── config.py           # Application settings
├── tests/
│   ├── __init__.py
│   └── test_books.py       # API endpoint tests
├── main.py                 # Application entry point
├── requirements.txt        # Project dependencies
└── README.md

Prerequisites

Docker. To install docker, check their official documentation.

Setting Up

The initial step was cloning the provided git repo, which contained the basic project structure. Here's a link to the project's git repo, https://github.com/NonsoEchendu/fastapi-book-project.

To clone:

git clone https://github.com/NonsoEchendu/fastapi-book-project.git

Challenge Task #1

So starting with the first task in the challenge: Add an endpoint to retrieve a book by its ID.

To do this, I need to go to the books.py file found in the routes/ folder, and add the /api/v1/books/{book_id} endpoint.

I'm also going to add a get_book function to retrieve the book details by ID. And if the book is not found, the function throws a 404 Not Found error.

Endpoint definition:

@router.get(
    "/{book_id}", response_model=Book, status_code=status.HTTP_200_OK
)
async def get_book(book_id: int) -> Book:
    book = db.books.get(book_id)
    if not book:
        raise HTTPException(status_code=404, detail="Book not found")
    return book

Next, i tested if the endpoint I just added is working.

To test, i'll be starting the application using Uvicorn:

uvicorn app.main:app --reload

Then, access the endpoint on my browser url at http://127.0.0.1:8000/api/v1/books/1.

It works!

Dockerizing the Application

Before moving to the next tasks for the challenge - creating CI/CD pipelines, i'll want to containerize the app using docker.

So first, i'll be creating a Dockerfile in the project root's directory. The Dockerfile will be structured to install dependencies, set up a non-root user, and start the app using Uvicorn.

Here's the Dockerfile:

FROM python:3.9-slim

WORKDIR /app

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 8000

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Now, we can build the application into a docker image and run it inside a container while maintaining a clean and isolated environment.

To build the image, while in the project's root directory, run:

docker build -t fastapi-book .

To run the app in a container:

docker run -d -p 8000:8000 fastapi-book

You should be able to also access it on your browser url at http://127.0.0.1:8000/api/v1/books/1.

Challenge Task #2

Moving to the next task: Setting up a CI (Continuous Integration) pipeline.

This CI pipeline will simply automate a pytest whenever a pull request is made to the main branch. These tests are important as they ensure that any changes to the codebase do not break existing functionality.

The CI pipeline:

name: CI Pipeline

on:
  pull_request:
    branches:
      - main

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout Repository
        uses: actions/checkout@v3

      - name: Set Up Python
        uses: actions/setup-python@v4
        with:
          python-version: "3.9"

      - name: Install Dependencies
        run: |
          python -m venv venv
          source venv/bin/activate
          pip install --upgrade pip
          pip install -r requirements.txt

      - name: Run Tests
        run: |
          source venv/bin/activate
          pytest --maxfail=1 --disable-warnings

Challenge Task #3

Setting up a CD (Continuous Deployment) pipeline

This CD pipeline will automate the deployment process whenever changes are pushed to the main branch.

The CD pipeline:

name: Deployment Pipeline

on:
  push:
    branches:
      - main

jobs:
  deploy:
    runs-on: ubuntu-latest

    env:
          SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
          SERVER_IP: ${{ secrets.SERVER_IP }}
          USER: ${{ secrets.SERVER_USER }}

    steps:
      - name: Checkout Repository
        uses: actions/checkout@v3

      - name: Debug Environment Variables
        run: |
          echo "Server IP: $SERVER_IP"
          echo "User: $USER"

      - name: Set Up SSH
        run: |
          # Setup SSH
          mkdir -p ~/.ssh 
          echo "$SSH_PRIVATE_KEY" | tr -d '\r' > ~/.ssh/id_rsa
          chmod 600 ~/.ssh/id_rsa
          ssh-keyscan -H $SERVER_IP >> ~/.ssh/known_hosts

      - name: Prepare EC2 Instance & Pull Latest Code
        run: |
          ssh $USER@$SERVER_IP << 'EOF'
            set -e  # Stop script if any command fails

            # Ensure project directory exists
            if [ ! -d "/home/$USER/fastapi-book-project" ]; then
              git clone https://github.com/NonsoEchendu/fastapi-book-project.git /home/$USER/fastapi-book-project
            fi

            # Go to project folder
            cd /home/$USER/fastapi-book-project

            # Fetch latest changes
            git reset --hard  # Ensure a clean state
            git pull origin main
          EOF

      - name: Install Docker
        run: |
          ssh $USER@$SERVER_IP << 'EOF'
            sudo apt-get update
            sudo apt-get install -y docker.io 
            sudo usermod -aG docker $USER
          EOF

          ssh $USER@$SERVER_IP "docker ps"

      - name: Deploy With Docker 
        run: |
          ssh $USER@$SERVER_IP << 'EOF'
            cd /home/$USER/fastapi-book-project
            docker stop fastapi-app || true
            docker rm fastapi-app || true
            docker build -t fastapi-app .
            docker run -d --name fastapi-app -p 8000:8000 fastapi-app
          EOF

This setup ensures that changes pushed to the main branch were automatically pulled into the server. It also builds the image and runs the application in a Docker container.

The use of secrets also ensures secure authentication without exposing sensitive credentials.

To setup the repository secrets in your Github Actions, go to the github repository settings. Then go to **Secrets and variables**, then click on **Actions**. Next, click on **New repository secret** and add the 3 repo secrets we used in the CD pipeline, namely, SERVER_IP, SERVER_USER, SSH_PRIVATE_KEY.

Before creating these secrets, please ensure to already create an AWS ec2 instance. The values assigned to these secrets will be from the running ec2 instance.

Challenge Task #4

And now the last task - Serving the application over NGINX.

I want to use NGINX as a reverse proxy to route traffic from my ec2 instance's public ip address to my FastAPI application running on port 8000.

Inside my ec2 instance ubuntu server, i'll first install and enable Nginx using:

sudo apt-get update

sudo apt-get nginx

sudo systemctl start enable nginx

Then i add the Nginx configuration file:

sudo nano etc/nginx/sites-available/fastapi-app

And add this:

server {
    listen 80;
    server_name yourdomain.com;  # Replace with your domain or server IP

    location / {
        proxy_pass http://127.0.0.1:8000/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Next, i enable the configuration by creating a symlink, and remove the default site:

sudo ln -s /etc/nginx/sites-available/fastapi /etc/nginx/sites-enabled/

sudo rm /etc/nginx/sites-enabled/default

Then i can restart NGINX:

sudo systemctl restart nginx

Now i can access the endpoint directly without adding 8000 to the url, like this: http://ec2-public-ip/api/v1/books/{book_id}.
{book_id} here being a positive integer.

Conclusion

In summary, what was I able to achieve in this project?

I built a FastAPI-based book API that retrieves book details by ID. I also made the application production-ready by containerizing it using Docker and used Nginx as a reverse proxy.

I also implemented CI/CD pipelines using GitHub Actions ensuring that the application is thoroughly tested and automatically deployed whenever changes are pushed to the main branch.

If you’re interested in exploring the code, check out the GitHub repository. Feel free to fork it, experiment, and adapt it to your needs!

Till my next project, happy building! ✌🏽

Speedy emails, satisfied customers

Are delayed transactional emails costing you user satisfaction? Postmark delivers your emails almost instantly, keeping your customers happy and connected.