DEV Community: Erick Okal

Building a Meeting Summarizer Backend with Python FastAPI, AWS Transcribe and AWS Bedrock

Erick Okal — Fri, 07 Mar 2025 13:31:10 +0000

Introduction

In this tutorial, we’ll build a meeting summarizer backend using FastAPI and AWS Transcribe and Bedrock foundational models. The application transcribes audio recordings, extracts key discussion points, and provides structured summaries with sentiment analysis and issue detection.

Key Features

Audio Transcription – Uses AWS Transcribe to convert speech to text.
Speaker Labeling – Identifies different speakers in the conversation.
Summarization – AWS Bedrock’s Titan model extracts key insights.
Sentiment Analysis & Issue Detection – Provides a concise summary with tone detection.
FastAPI Backend – A lightweight, high-performance API for seamless integration.

Tech Stack

FastAPI – Lightweight web framework for Python
AWS Transcribe – Speech-to-text conversion
AWS Bedrock – Fully managed AI service providing LLM integration
Amazon S3 – Cloud storage for audio files and transcriptions
Jinja2 – Template engine for prompt formatting

Step 1: Project Setup

1. Install Prerequisites

Python 3.10+
Poetry 1.8+ – Dependency management tool
AWS CLI (Optional, for testing)

2. AWS S3 and Bedrock setup

Create two s3 buckets and grant necessary permissions.
- AWS_BUCKET_NAME - Bucket for holding the Audio files
- OUTPUT_BUCKET_NAME - Bucket for holding transcriptions
Request model access. In this example I'm using Titan Text G1 - Lite.

3. Clone the Repository

git clone https://github.com/bokal2/meeting-summarizer-backend.git
cd meeting-summarizer-backend

4. Install Dependencies

install dependencies:

poetry shell
poetry install

4. Configure AWS Credentials

Create a .env file with the following:

AWS_REGION=your_aws_region
AWS_ACCESS_KEY_ID=your_access_key
AWS_SECRET_ACCESS_KEY=your_secret_key
AWS_BUCKET_NAME=your_bucket_name
OUTPUT_BUCKET_NAME=your_output_bucket_name

Step 2: API Implementation

Main Components

The backend consists of:

Audio Upload & Transcription – Sends audio files to AWS S3 and triggers AWS Transcribe.
Text Processing – Converts transcribed text into a structured format.
Summarization with AWS Bedrock – Generates meeting summaries based on a prompt template.

FastAPI Implementation (main.py)

import json
import time
import uuid
from fastapi import FastAPI, HTTPException, File, UploadFile
from fastapi.templating import Jinja2Templates
from fastapi.middleware.cors import CORSMiddleware
import boto3
from decouple import config

AWS_REGION = config("AWS_REGION")
AWS_ACCESS_KEY_ID = config("AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = config("AWS_SECRET_ACCESS_KEY")
BUCKET_NAME = config("AWS_BUCKET_NAME")
OUTPUT_BUCKET_NAME = config("OUTPUT_BUCKET_NAME")

app = FastAPI()

# Configure allowed origins
origins = [
    "http://localhost:3000", # Testing with a React App
]

app.add_middleware(
    CORSMiddleware,
    allow_origins=origins,
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

templates = Jinja2Templates(directory="templates")

async def upload_file_to_s3(file_obj, file_name, s3_client, bucket_name):
    try:
        s3_client.upload_fileobj(file_obj, bucket_name, file_name)
    except Exception as e:
        raise HTTPException(
            status_code=400,
            detail=f"File uplaod failed: {e}",
        )

def process_transcription(transcript_json):
    output_text = ""
    current_speaker = None

    items = transcript_json["results"]["items"]

    for item in items:

        speaker_label = item.get("speaker_label", None)
        content = item["alternatives"][0]["content"]

        if speaker_label is not None and speaker_label != current_speaker:
            current_speaker = speaker_label
            output_text += f"\n{current_speaker}: "

        if item["type"] == "punctuation":
            output_text = output_text.rstrip()

        output_text += f"{content} "

    return output_text

async def transcribe_audio(
    model_id,
    bucket_name,
    file_name,
    file_content,
    output_bucket,
):

    # Upload Audio to s3 bucket
    s3_client = boto3.client(
        "s3",
        region_name=AWS_REGION,
        aws_access_key_id=AWS_ACCESS_KEY_ID,
        aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
    )
    await upload_file_to_s3(
        file_obj=file_content,
        file_name=file_name,
        s3_client=s3_client,
        bucket_name=bucket_name,
    )

    transcribe_client = boto3.client(
        "transcribe",
        region_name=AWS_REGION,
        aws_access_key_id=AWS_ACCESS_KEY_ID,
        aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
    )

    job_name = f"transcription-job-{uuid.uuid4()}"

    transcribe_client.start_transcription_job(
        TranscriptionJobName=job_name,
        Media={"MediaFileUri": f"s3://{bucket_name}/{file_name}"},
        MediaFormat="mp3",
        LanguageCode="en-US",
        OutputBucketName=output_bucket,
        Settings={"ShowSpeakerLabels": True, "MaxSpeakerLabels": 2},
    )

    while True:
        job_status = transcribe_client.get_transcription_job(
            TranscriptionJobName=job_name,
        )

        status = job_status["TranscriptionJob"]["TranscriptionJobStatus"]
        if status in ["COMPLETED", "FAILED"]:
            break
        time.sleep(2)

    if status == "FAILED":
        raise HTTPException(status_code=400, detail="Transcription Job failed")

    transcript_key = f"{job_name}.json"
    transcript_obj = s3_client.get_object(
        Bucket=output_bucket,
        Key=transcript_key,
    )
    transcript_text = transcript_obj["Body"].read().decode("utf-8")
    transcript_json = json.loads(transcript_text)

    output_text = process_transcription(transcript_json)

    result = await summarize_transcription(
        model_id,
        transcript=output_text,
    )

    return result

Step 3: Summarization Using AWS Bedrock

Prompt Engineering

We use a Jinja2 template to format the transcript for the Bedrock model:

I need to analyze and summarize a conversation. The transcript of the
conversation is between the <data> XML-like tags.

<data>
{{transcript}}
</data>

Please do the following:
1. Identify the main topic being discussed.
2. Provide a concise summary of key points.
3. Include a one-word sentiment analysis.
4. List any issues, problems, or conflicts.

Format the output in JSON:
{
    "topic": "<main_topic>",
    "meeting_summary": "<summary>",
    "sentiment": "<one_word_sentiment>",
    "issues": [{"topic": "<issue>", "summary": "<description>"}]
}

AWS Bedrock Summarization

async def summarize_transcription(model_id: str, transcript: str):

    bedrock_runtime = boto3.client(
        "bedrock-runtime",
        region_name=AWS_REGION,
        aws_access_key_id=AWS_ACCESS_KEY_ID,
        aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
    )

    template = templates.get_template("prompt_template.txt")

    rendered_prompt = template.render(transcript=transcript)

    try:
        kwargs = {
            "modelId": model_id,
            "contentType": "application/json",
            "accept": "*/*",
            "body": json.dumps(
                {
                    "inputText": rendered_prompt,
                    "textGenerationConfig": {
                        "maxTokenCount": 512,
                        "temperature": 0,
                        "topP": 0.9,
                    },
                }
            ),
        }
        # Call AWS Bedrock
        response = bedrock_runtime.invoke_model(**kwargs)
        # Parse response
        response_body = json.loads(response.get("body").read())
        result = response_body["results"][0]["outputText"]
        return {"response": result}
    except Exception as e:
        raise HTTPException(
            status_code=500,
            detail=f"Error invoking Bedrock: {str(e)}",
        )

Summary API Endpoint

@app.post("/summary")
async def audio_summary_test(
    model_id: str = "amazon.titan-text-lite-v1",
    file: UploadFile = File(...),
):
    """An endpoint for generating meeting summary from audio file"""
    # But first, ensure the cursor is at position 0:
    file.file.seek(0)

    response = await transcribe_audio(
        model_id=model_id,
        bucket_name=BUCKET_NAME,
        file_name=file.filename,
        file_content=file.file,
        output_bucket=OUTPUT_BUCKET_NAME,
    )

    return {"response": response}

Step 4: Running the Application

1. Start the FastAPI Server

uvicorn main:app --reload

2. Test the API Using cURL

curl -X POST "http://127.0.0.1:8000/summary" \
-H "Content-Type: multipart/form-data" \
-F "file=@meeting_audio.mp3" \
-F "model_id=amazon.titan-text-lite-v1"

3. Sample JSON Response

{
    "response": {
        "topic": "Project Updates",
        "meeting_summary": "The meeting discussed progress on Q1 deliverables...",
        "sentiment": "positive",
        "issues": [
            {"topic": "Timeline Delay", "summary": "The team noted delays in the design phase."}
        ]
    }
}

4. Test the API Using Next.js Application

I created a simple Next.js app to test the API. You can find the code in this Git repository, along with detailed setup instructions in the README to help you get it up and running quickly.

Some Noticeable Challanges

Accuracy in Transcription – Issues with accents, low audio volume, overlapping speech, and background noise can lead to poor transcription results.
LLM Summarization Accuracy – May miss nuances or oversimplify complex discussions.
Processing Time and Latency – Large audio files lead to long transcription times and LLM response delays.
Scalability Issues – Handling multiple users and large audio files can lead constraints on the underlying resources.
Prompt Engineering Complexity – Designing effective prompts for sequential or chat-based interactions is challenging, with limited reference resources currently available.

Conclusion

Exploring AWS Bedrock and experimenting with different foundation models was an exciting experience. It’s impressive how seamlessly developers can leverage these models to build LLM-powered applications with minimal hassle. The potential is immense, and I look forward to diving deeper, exploring advanced models, and uncovering new possibilities.

Next Steps:

Deploy the API using AWS Lambda or EKS
Enhance prompt engineering for better summarization accuracy

Tail Kubernetes Logs Efficiently with Stern

Erick Okal — Wed, 19 Feb 2025 14:10:30 +0000

Monitoring logs in Kubernetes is essential for debugging and troubleshooting. While kubectl logs -f pod-name works for individual pods, it becomes difficult when dealing with multiple pods in a deployment. Sometimes we end up opening multiple terminal tabs to follow logs. Well, stern can be a better alternative.

Why Use Stern?

Allows tailing logs from multiple pods simultaneously
Supports regex filtering for better log analysis
Works with multi-container pods
Provides color-coded output for better readability

Installing Stern

macOS: brew install stern
Linux:

curl -Lo stern 
https://github.com/wercker/stern/releases/download/{VERSION}/stern_linux_amd64
chmod +x stern
sudo mv stern /usr/local/bin/

Note:

To illustrate some of the commands, I'll use meetings-summarizer as my deployment.

Using Stern for Log Tailing

Tail logs from all pods in a deployment

  stern meetings-summarizer

Tail logs from a specific container in a pod

  stern meetings-summarizer -c backend

Follow logs from the deployment pods across all namespaces

  stern meetings-summarizer --all-namespaces

Filter logs using regex

  stern meetings-summarizer | grep "ERROR"

Exclude unwanted log patterns

  stern meetings-summarizer --exclude "healthcheck"

Enable colorized output for better readability

  stern meetings-summarizer --color=always

When to Use Stern?

Debugging microservices that generate logs across multiple pods
Monitoring real-time logs for deployments
Filtering logs for specific containers or error messages
Analyzing logs in large-scale Kubernetes environments

Stern simplifies log management in Kubernetes, making it easier to diagnose and resolve issues efficiently. I hope you find this article useful.

AWS Lambda: The Pros, Cons, and When to Use It (or Not)

Erick Okal — Sun, 19 Jan 2025 19:05:36 +0000

In recent years serverless architectural patterns have gained a lot of traction. AWS offers several serverless services and AWS lambda is one of the popular ones and an example of a Function as a Service(FaaS) serverless architecture.

What is AWS Lambda?

AWS Lambda is a serverless compute service to let you execute your code to do a specialized task without having to provision or manage servers. You can upload your code to AWS and define the runtime configuration, CPU and memory requirements, and an event on which your function should be executed. AWS is responsible for booting up servers and required runtime environments to execute the code you provided as soon as the triggering event takes place.

Lambda function workflow diagram:

Cost consideration and breakdown

The Non-expiry free tier includes 1M invocations and 400,000 GB-seconds of compute per month.
Charges are based on 1 ms increments, ensuring you only pay for what you use
You never pay for idle resources, making it ideal for sporadic workloads.
AWS Pricing Calculator can help estimate costs based on your workload."*

Pros of AWS Lambda:

AWS Lambda offers several distinct advantages over maintaining your own servers in the cloud:

Pay Per Use: With AWS Lambda, you only pay for the compute time your functions use and any network traffic generated. This billing model is especially cost-effective for workloads that vary significantly based on time of day or demand.
Fully Managed Infrastructure: AWS handles all aspects of the underlying infrastructure, such as operating system updates and network management. This eliminates the need for operational tasks, saving time and reducing complexity.
Automatic Scaling: Lambda automatically scales your functions to handle varying loads, creating instances as requests come in. There’s no need to predefine scale levels or manage scaling settings, your functions simply scale up or down based on demand, and you only pay for runtime.
Seamless Integration with AWS Services: Lambda integrates tightly with other AWS products, such as DynamoDB, S3, and API Gateway, enabling you to build complete applications with minimal effort. This integration streamlines workflows and simplifies the development of serverless applications.

Cons of AWS Lambda

While AWS Lambda has many advantages, there are also some limitations to consider:

Cold Start Latency: Functions not used in the last 15 minutes may experience a cold start introducing a 5-10 second delay, making Lambda unsuitable for latency-critical applications.
Execution Time Limits: Lambda functions have a maximum runtime of 15 minutes, which cannot be extended. This limitation makes it unsuitable for long-running tasks like video rendering or large-scale data processing.
Resource Constraints: Functions are limited to 10GB of memory, 6 vCPUs, and 10GB of ephemeral storage. These constraints can hinder tasks requiring high computational power or large datasets.
Stateless Nature: Lambda functions are stateless by design, requiring external services like DynamoDB or S3 for state management. This can add complexity for workflows that require persistent in-memory data.
Cost at Scale: Although cost-efficient for low-to-moderate workloads, Lambda's pay-per-use model can become expensive for high-frequency or resource-intensive workloads, where EC2 or ECS may be more economical.

Conclusion:

AWS Lambda is a powerful serverless tool when used in the right scenarios. By understanding its strengths and limitations, you can harness it to build scalable, efficient, and cost-effective applications. Have you used Lambda in your recent projects? It would be nice to hear your experiences!

Some useful resources:

Project #02: Building a 3-Tier AWS VPC Architecture Using Terraform

Erick Okal — Wed, 18 Dec 2024 16:25:26 +0000

In this project, we’ll dive into building a 3-tier AWS VPC architecture using Terraform. This architecture demonstrates how to span resources across multiple Availability Zones (AZs), create both public and private subnets, and configure NAT Gateways to enable secure internet access (e.g downloading software updates) for instances in private subnets via the Internet Gateway.

Prerequisites

AWS Account: Active AWS account with permissions to create resources like VPCs, subnets, and EC2 instances.
Terraform Installed: Install Terraform from the official website.
AWS CLI

Note on Costs

Deploying this architecture will incur AWS costs, even when using free-tier eligible resources, due to the following:

Elastic IPs: Costs associated with each NAT Gateway.
NAT Gateways: Charges for usage and data transfer.

Tip: Use the AWS Pricing Calculator to estimate costs.

Architectural Diagram

Project Steps

Here’s a breakdown of how the 3-tier AWS VPC architecture was created.

Create the Network Infrastructure

A VPC was set up with a CIDR block of 10.0.0.0/16 to host all the resources.
Public subnets were created in two Availability Zones (AZs) to host NAT Gateways and allow public-facing resources to access the internet.
Private subnets were configured:
- Application subnets to host the application instances.
- Database subnets for securely hosting database instances.
An Internet Gateway was attached to the VPC to provide internet connectivity for resources in public subnets.
NAT Gateways were deployed in the public subnets to enable secure internet access for private subnets, allowing both application and database instances to download software updates and libraries from the internet.
Route tables were created and associated with the subnets:
- Public subnets routed traffic directly through the Internet Gateway.
- Private subnets routed outbound traffic through the NAT Gateways.

Clean Up Resources

Terraform’s destroy command ensured all resources (VPC, subnets, NAT Gateways, instances, and route tables) were systematically deleted once the deployment was no longer needed.

Terraform Implementation

Here’s the Terraform configurations used to set up the VPC, subnets, NAT gateways and route tables.

1. Providers and Variables

providers.tf:

terraform {
  required_version = ">= 1.7.5"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = var.aws_region
}

variables.tf:
Defined key variables for modularity and flexibility.

variable "project_name" {
  type        = string
  description = "The name of the project"
  default     = "3-tier-architecture"
}

variable "aws_region" {
  type        = string
  description = "The AWS region to create resources in"
  default     = "us-east-1"
}

variable "azs" {
  type        = list(string)
  description = "The availability zones to create resources in"
  default     = ["us-east-1a", "us-east-1b"]
}

variable "vpc_cidr" {
  type        = string
  description = "The CIDR block for the VPC"
  default     = "10.0.0.0/16"
}

variable "public_subnets" {
  type        = list(string)
  description = "The public subnets to create"
  default     = ["10.0.1.0/24", "10.0.2.0/24"]
}

variable "app_subnets" {
  type        = list(string)
  description = "The private subnets to host the application"
  default     = ["10.0.3.0/24", "10.0.4.0/24"]
}

variable "db_subnets" {
  type        = list(string)
  description = "The private subnets to host the database"
  default     = ["10.0.5.0/24", "10.0.6.0/24"]
}

2. VPC and Subnets

network.tf:

locals {
  common_tags = {
    Project = var.project_name
  }
}

# VPC
resource "aws_vpc" "main" {
  cidr_block           = var.vpc_cidr
  enable_dns_support   = true
  enable_dns_hostnames = true
  tags                 = merge(local.common_tags, { Name = "${var.project_name}-vpc" })
}

# Public Subnets
resource "aws_subnet" "public" {
  for_each                = toset(var.public_subnets)
  vpc_id                  = aws_vpc.main.id
  cidr_block              = each.key
  availability_zone       = element(var.azs, index(var.public_subnets, each.key))
  map_public_ip_on_launch = true
  tags                    = merge(local.common_tags, { Name = "public-subnet-${each.key}" })
}

# App Subnets
resource "aws_subnet" "app-subnet" {
  for_each          = toset(var.app_subnets)
  vpc_id            = aws_vpc.main.id
  cidr_block        = each.key
  availability_zone = element(var.azs, index(var.app_subnets, each.key))
  tags              = merge(local.common_tags, { Name = "app-subnet-${each.key}" })
}

# DB Subnets
resource "aws_subnet" "db-subnet" {
  for_each          = toset(var.db_subnets)
  vpc_id            = aws_vpc.main.id
  cidr_block        = each.key
  availability_zone = element(var.azs, index(var.db_subnets, each.key))
  tags              = merge(local.common_tags, { Name = "db-subnet-${each.key}" })
}

3. Internet Gateway and NAT Gateways

nat_gateway.tf:

# Internet Gateway
resource "aws_internet_gateway" "igw" {
  vpc_id = aws_vpc.main.id
  tags   = merge(local.common_tags, { Name = "${var.project_name}-igw" })
}

# Elastic IPs for NAT Gateways
resource "aws_eip" "nat_eip" {
  for_each = toset(var.azs)
  tags     = merge(local.common_tags, { Name = "nat-eip-${each.key}" })
}

# NAT Gateways
resource "aws_nat_gateway" "nat" {
  for_each      = toset(var.azs)
  allocation_id = aws_eip.nat_eip[each.key].id
  subnet_id     = aws_subnet.public[element(var.public_subnets, index(var.azs, each.key))].id

  tags = merge(local.common_tags, { Name = "nat-gateway-${each.key}" })
}

4. Route Tables

routes.tf:

# Public Route Table
resource "aws_route_table" "public" {
  vpc_id = aws_vpc.main.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.igw.id
  }

  tags = {
    Name = "public-rt"
  }
}

# Associate Public Subnets with Public Route Table
resource "aws_route_table_association" "public" {
  for_each = aws_subnet.public
  subnet_id      = each.value.id
  route_table_id = aws_route_table.public.id
}

# Private Route Tables for App and DB Subnets
resource "aws_route_table" "private" {
  for_each = toset(var.azs)
  vpc_id = aws_vpc.main.id

  route {
    cidr_block     = "0.0.0.0/0"
    nat_gateway_id = aws_nat_gateway.nat[each.key].id
  }

  tags = {
    Name = "private-rt-${each.key}"
  }
}

# Associate Private Subnets with Private Route Tables
resource "aws_route_table_association" "app" {
  for_each = aws_subnet.app
  subnet_id      = each.value.id
  route_table_id = element(aws_route_table.private[*].id, index(var.app_subnets, each.key))
}

resource "aws_route_table_association" "db" {
  for_each = aws_subnet.db
  subnet_id      = each.value.id
  route_table_id = element(aws_route_table.private[*].id, index(var.db_subnets, each.key))
}

Next Steps

Deploy application EC2 instances in the private application subnets to host the application layer.
Set up a database instance (e.g., Amazon RDS) in the private database subnets.
Integrate an Elastic Load Balancer (ELB) and Auto Scaling Group (ASG) to ensure high availability and scalability for instances in the application layer.

Try It Yourself

Clone the repository:

git clone https://github.com/bokal2/terraform-projects.git
cd 3-tier-architecture

Follow the steps in README to deploy resources.

Thank you and please feel free to share your thoughts. Cheers!

Architecting AWS with Terraform Series.

Erick Okal — Tue, 10 Dec 2024 12:44:51 +0000

Infrastructure as Code (IaC) has transformed the way developers and operations teams provision and manage infrastructure, enabling automation, repeatability, and scalability. Recently, I completed a significant project involving the migration of workloads from on-premises to the AWS cloud. Along the way, I learnt a ton and gained skills that I’m excited to share.

To document and reflect on these lessons, I’ve decided to launch a new series highlighting the key insights and skills acquired through these projects.

Project #01: Deploying an NGINX Server on AWS with Terraform

This is a simple project just to demonstrate how easy and convenient we can deploy an NGINX server on AWS using Terraform. It showcases how automation and repeatable processes can simplify infrastructure provisioning while ensuring consistency and reliability.

Project Steps

Here’s a breakdown of how the infrastructure was created:

1. Create the Network Infrastructure

A VPC was set up with a CIDR block of 10.0.0.0/16.
A public subnet within the VPC allowed instances to connect to the internet.
An Internet Gateway was attached to the VPC for external connectivity.
A route table was created and associated with the subnet to route traffic through the Internet Gateway.

2. Configure Security Groups

A security group was defined to allow:
- Inbound traffic on port 80 (HTTP).
- Inbound traffic on port 443 (HTTPS).
Outbound traffic was unrestricted to enable communication with external resources.

3. Deploy the EC2 Instance

A t2.micro instance was launched using a publicly available Ubuntu AMI.
The instance was configured with a public IP address to allow external access.
A user data script automated the installation and setup of the NGINX web server:

  #!/bin/bash
  sudo apt-get update -y
  sudo apt-get install -y nginx
  sudo systemctl start nginx
  sudo systemctl enable nginx

4. Verify the Deployment

Terraform outputs included the public IP address of the EC2 instance.
The default NGINX page was accessible by navigating to the public IP in a browser.

5. Clean Up Resources

Terraform’s destroy command ensured all resources were safely and systematically deleted when no longer needed.

Key Project Files

providers.tf: Configured the AWS provider and defined the Terraform version:

terraform {
   required_version = ">= 1.7.5"

   required_providers {
     aws = {
       source  = "hashicorp/aws"
       version = "~> 5.0"
     }
   }
 }

 provider "aws" {
  region = "us-east-1"
 }

network.tf: Defined the VPC, subnet, Internet Gateway, route table, and associations:

locals {
  common_tags = {
    project = "project01"
  }
}

resource "aws_vpc" "project01-vpc" {
  cidr_block = "10.0.0.0/16"
  tags = merge(local.common_tags, {
    Name = "project01-vpc"
  })
}

resource "aws_subnet" "public-subnet" {
  vpc_id     = aws_vpc.project01-vpc.id
  cidr_block = "10.0.0.0/24"
  tags = merge(local.common_tags, {
    Name = "public-subnet"
  })
}

resource "aws_internet_gateway" "project01-igw" {
  vpc_id = aws_vpc.project01-vpc.id

  tags = merge(local.common_tags, {
    Name = "project01-igw"
  })
}

resource "aws_route_table" "project01-rtb" {
  vpc_id = aws_vpc.project01-vpc.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.project01-igw.id
  }

  tags = merge(local.common_tags, {
    Name = "project01-rtb"
  })
}

resource "aws_route_table_association" "public" {
  subnet_id      = aws_subnet.public-subnet.id
  route_table_id = aws_route_table.project01-rtb.id
}

compute.tf: Provisioned the EC2 instance and security group:


resource "aws_security_group" "nginx-server-sg" {
  description = "Security group allowing HTTP(port 80) and HTTPS(port 443)"
  name        = "nginx-server-sg"
  vpc_id      = aws_vpc.project01-vpc.id

  # Allow inbound HTTP traffic on port 80
  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  # Allow inbound HTTP traffic on port 443
  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  # Allow all outbound traffic
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]

  }

  tags = merge(local.common_tags, {
    Name = "nginx-server-sg"
  })

}

resource "aws_instance" "nginx-server" {
  ami                         = "ami-0866a3c8686eaeeba"
  associate_public_ip_address = true
  instance_type               = "t2.micro"
  subnet_id                   = aws_subnet.public-subnet.id
  vpc_security_group_ids      = [aws_security_group.nginx-server-sg.id]


  root_block_device {
    delete_on_termination = true
    volume_size           = 10
    volume_type           = "gp3"
  }

  user_data = <<-EOF
                #!/bin/bash
                sudo apt-get update -y
                sudo apt-get install -y nginx
                sudo systemctl start nginx
                sudo systemctl enable nginx
                EOF

  tags = merge(local.common_tags, {
    Name = "project01-nginx-server"
  })

  lifecycle {
    create_before_destroy = true
  }
}

Conclusion

This project offers a glimpse into the capabilities of Infrastructure as Code (IaC) tools like Terraform. While it’s neither exhaustive nor flawless, it serves as a foundation for exploring how IaC can simplify and automate infrastructure management. I hope this project sparks ideas and inspires you to dive deeper into IaC practices.

Your feedback is invaluable, please feel free to share your thoughts. I’d love to hear from you!

Try It Yourself

Clone the repository:

git clone https://github.com/bokal2/terraform-projects.git
cd project01

Follow the steps in README to deploy your own NGINX server.

Happy building!

Embracing Infrastructure as Code (IaC): An Essential Practice for Modern Teams

Erick Okal — Tue, 03 Dec 2024 18:36:58 +0000

In today’s fast-evolving tech landscape, Infrastructure as Code (IaC) has emerged as a powerful practice for provisioning and managing cloud infrastructure through code rather than manual processes. Before diving into a practical tool like Terraform, it’s crucial to understand what IaC offers and why it’s a must-have for scaling reliable, secure, and repeatable infrastructure.

The Manual Past

Traditionally, deploying infrastructure often meant manually configuring resources through a provider's console (like AWS or GCP) or using the Command Line Interface (CLI). While effective for quick, small-scale deployments, this approach introduces significant challenges in modern environments where scale, speed, and reliability are key.

Challenges and Limitations

Complexity in Large-Scale Reproductions: As infrastructure grows, so does the need to replicate configurations for new projects, environments, or disaster recovery. With manual steps, reproducing a full setup with databases, virtual networks, and compute resources can be tedious, time-consuming, and error-prone.
Lack of Change Tracking: Without a centralized codebase, tracking changes across team members can be challenging. If someone modifies the infrastructure without notifying the team, it could lead to unforeseen issues or downtime.
Error-Prone Processes: Manual processes require detailed knowledge of specific CLI commands and order of operations, making it easy to misconfigure settings or make errors that only surface later, often in production environments.
Difficulty in Auditing and Rolling Back Changes: Without version control, auditing who changed what and why is a headache, and reverting changes becomes manual and risky.

The bottom line? Manual infrastructure management, while quick for experiments, lacks the control and consistency required for scalable, production-ready systems.

Infrastructure as Code:

Infrastructure as Code transforms this manual, error-prone approach by allowing you to describe your desired infrastructure in code. Instead of clicking through the console or writing individual CLI commands, IaC tools like Terraform allow you to define the end state of your infrastructure in a declarative way.

Benefits at Scale

Codified Infrastructure for Easy Reproduction: With all configurations written in code, reproducing infrastructure across environments becomes straightforward and reliable. You’re essentially telling the IaC tool what your infrastructure should look like, and the tool ensures everything is deployed as specified.
Centralized Change Tracking and Version Control: Storing IaC code in a version-controlled repository provides a full history of infrastructure changes, making it easy to track, audit, and revert configurations. It’s clear who made changes and when, streamlining team collaboration.
Declarative Approach, Not Imperative: With IaC, you only need to specify the desired state of resources, not the detailed steps to get there. This approach abstracts away underlying APIs and complex sequences, making infrastructure setup simpler and accessible to a broader range of team members.
Reduced Configuration Errors: IaC tools offer validations and checks before applying changes, catching configuration issues early. Many IaC processes integrate with CI/CD pipelines, automating validation and deployment steps to further minimize manual errors.
Enhanced Disaster Recovery: Since your entire infrastructure setup is codified, you can quickly replicate or redeploy resources in the event of a failure, supporting robust disaster recovery and continuity strategies.

How Terraform Helps Make IaC a Reality

Terraform, one of the most popular IaC tools, reads your infrastructure code and automates the process of building, updating, and managing your resources. Here’s how it works:

Declarative Configuration Language: In Terraform, you write configuration files that describe your desired end state. Terraform then takes these files, plans the required actions, and provides you with a detailed preview.
Automated Planning and Execution: Once you review and approve Terraform’s plan, it will apply these changes, creating, modifying, or deleting resources as needed. This helps you manage infrastructure at scale without needing to understand every detail of the underlying APIs.
Seamless Integration with CI/CD Pipelines: Terraform can be integrated into CI/CD pipelines, automating the infrastructure lifecycle and ensuring infrastructure is updated as part of the software delivery process.

Conclusion

Infrastructure as Code isn’t just a new way to deploy infrastructure; it’s a practice that brings control, efficiency, and reliability to your organization’s infrastructure lifecycle. With IaC tools like Terraform, you gain the power to deploy infrastructure quickly, manage it as code, and maintain full control over your infrastructure history.