DEV Community: Audu Ephraim

Building an FIA Formula 1 Financial Regulations Chatbot with AWS S3, Kendra and Bedrock

Audu Ephraim — Wed, 23 Apr 2025 11:33:56 +0000

This project was inspired by Mark Tinderholt’s post on creating a JFK assassination chatbot using the JFK files. You can find it Here. As a Formula 1 fan, I wanted to understand the sport’s complex financial regulations more, and what better way of doing that than building a project around it? So I built a chatbot that lets me query the official FIA Formula 1 Financial Regulations PDF available on the FIA official website using natural language.

With AWS Kendra for document indexing and Amazon Bedrock’s Titan model for answering questions in a conversational style, I was able to turn a static document into a dynamic knowledge bot.

🚀 What I Built

A Python-based CLI application that:

Queries AWS Kendra for relevant context.
Uses Bedrock’s Amazon Titan Text G1 -Express model to generate chat-like answers.

🧠 Why This Project?

I wanted to:

Understand how to combine semantic search (Kendra) with LLMs (Bedrock).
understand how LLMs products are built
Make a dense PDF document more explorable.

🧪 Technologies Used

AWS S3 – To store the source PDF.
Amazon Kendra – As the search engine to index and retrieve document snippets.
Amazon Bedrock – To generate natural language answers based on Kendra’s output.
boto3 – AWS SDK for Python.

📝 Python Script Overview

query_kendra(query_text, index_id) – Sends a query to Kendra and collects document excerpts.
query_bedrock(query, context) – Sends context and query to Bedrock Titan to get a conversational response.
kendra_bedrock_query(user_query, index_id) – Main function that stitches the two together.

🧱 Architecture Overview

CLI App (Python) → Kendra (document search) → Bedrock (contextual response)
S3 acts as the document repository
IAM roles/policies allow access to services

💡 What I Learned

Kendra doesn’t require Textract if the PDF text is extractable. I originally used Textract to index PDF files and store the index in S3 for Amazon Kendra to query. However, I later realized that Kendra can handle this process independently. Now, I only need Textract for specific cases, such as processing receipts or extracting text from images.
Bedrock’s Titan G1 Express can process large contexts and return quality responses. I would like to do this with a more powerful model to see the difference in responses. I felt some answers would have been better.
IAM permissions need to include both kendra, bedrock, logs, and s3 access for things to sync correctly.

🧾 Sample Question

Q: What is cost cap administration?

A: Cost Cap Administration is a process of cost control that involves setting a limit on the amount of money that can be spent on a project or activity. This limit is typically set by an external organization, such as a government agency or a regulatory body, and is designed to ensure that the project or activity is completed within the allocated budget. The Cost Cap Administration process typically involves the following steps:
1. Establishing the cost cap: The cost cap is set by the external organization that is responsible for managing the project or activity. This limit is typically based on a variety of factors, such as the expected costs of the project, the expected revenue, and the expected risks.
2. Monitoring the costs: The Cost Cap Administration team monitors the costs of the project or activity on a regular basis. This includes reviewing invoices, tracking expenses, and analyzing financial data.
3. Adjusting the costs: If the costs of the project or activity exceed the cost cap, the Cost Cap Administration team must take action to adjust the costs. This may include reducing the scope of the project, reducing the number of employees, or reducing the amount of time that the project is scheduled to take.
4. Reporting the costs: The Cost Cap Administration team must report the costs of the project or activity to the external organization that is responsible for managing the project or activity. This report must include a detailed breakdown of the costs, including the amount of money that was spent, the amount of money that was saved, and the reasons for the changes in costs.
5. Enforcing the cost cap: The Cost Cap Administration team must enforce the cost cap. This may include issuing fines or penalties to the project or activity if the costs exceed the cost cap.
6. Reviewing the cost cap: The Cost Cap Administration team must review the cost cap on a regular basis to ensure that it is still appropriate. This may include adjusting the cost cap based on changes in the project or activity, or adjusting the cost cap based on feedback from the external organization that is responsible for managing the project or activity.

In conclusion, the Cost Cap Administration process is a critical process for ensuring that a project or activity is completed within the allocated budget. It involves establishing the cost cap, monitoring the costs, adjusting the costs, reporting the costs, enforcing the cost cap, and reviewing the cost cap on a regular basis. By following these steps, the Cost Cap Administration team can help to ensure that the project or activity is successful and that the external organization that is responsible for managing the project or activity is satisfied with the results.

Q: How much is a team eligible to spend each year?

A: $15,000,000

Q: What is the punishment for cost cap breach?

A: Based on the provided content, the punishment for cost cap breach is sanctions.

Q: What are the sanctions for cost cap breach?

A: Based on the provided content, the sanctions for cost cap breach include a financial penalty.

📌 Future Improvements

Build a full web app around it
Support multi-document uploads. This would become a full F1 rules and regulations chatbot.
Save chat history

🔗 GitHub Repo

https://github.com/audu97/FIA-F1-financial-reg/tree/main

Thanks for reading! Feel free to fork the project, open issues, or suggest new directions!

End-to-End workflow with Azure Devops, Docker and ArgoCD

Audu Ephraim — Sat, 17 Aug 2024 05:47:17 +0000

This project demonstrates how an end-to-end Gitops workflow was automated from start to finish using

Azure devops pipeline to build an application into a docker image and push said image to DockerHub.
configure Argocd to watch repository Kubernetes configuration files and update a Kubernetes cluster based on Kubernetes configuration files.

Azure DevOps Pipeline

trigger:
  branches:
    include:
    - master
  paths:
    exclude:
    - 'k8/*'

resources:
- repo: self

variables:
  dockerHub: 'docker-hub-service'  
  imageName: 'ephraimaudu/azure-argo-dev'
  imageTag: '$(Build.BuildId)'

stages:
  - stage: Build
    displayName: build and push image to docker hub
    jobs:
      - job: Build
        displayName: build
        pool: 
          name: myagent
        steps:
        - task: Docker@2
          inputs:
            containerRegistry: $(dockerHub)
            repository: $(imageName)
            command: 'buildAndPush'
            Dockerfile: '**/Dockerfile' 
            tags: |
              $(imageTag)

build is triggered when a change is made in the master branch excluding files in the k8 folder. those are Kubernetes configuration files
The Azure devops pipeline was configured to run on a self-hosted agent called "myagent" while running on the local machine. installation and configuration guide is pretty straightforward and can be found Here

ArgoCD

ArgoCD is a continuous delivery tool for Kubernetes. It automates application deployment and lifecycle management from Git repositories to specified clusters.

Essentially, it ensures that your live application state matches the desired state defined in your Git repository.

The kubernetes configuration file in the specified repository is constantly being watched, when they change argocd compares with the live application in the cluster if they are out of sync argocd syncs the cluster to match the configuration files.

ArgoCD runs in its own namespace running its pods

Argocd provides a detailed web UI to manage cluster or clusters

link to repository

using Ansible to deploy an application

Audu Ephraim — Sat, 10 Aug 2024 08:35:11 +0000

This is a simple Ansible script to install packages, and configurations and to deploy a simple go application on an Azure VM.

it installs nginx
installs golang
configure nginx and its services
and finally copies the go application binary from the local machine to vm repository contains the various scripts

Send Slack Notifications with Go AWS Lambda Functions

Audu Ephraim — Mon, 08 Jul 2024 23:51:53 +0000

Introduction

In this article, we will discuss how to create an AWS lambda function to send Slack notifications when the CPU utilization of an AWS instance reaches 50%.

AWS Lambda is a serverless compute service offered by Amazon Web Services (AWS). It lets you run code without having to provision or manage servers yourself.

It is event-driven i.e. your code executes in response to events triggered by other AWS services like a file upload completed in s3, an HTTP request from Amazon API Gateway or various other triggers.

In this, we will be discussing how to set up Amazon Cloudwatch to monitor and collect metrics from an EC2 instance, Cloudwatch alarms based on those metrics to trigger a notification when a certain threshold or condition is met, Amazon Simple Notification service to receive these notifications and finally a lambda function subscribed to the SNS topic, which will process the notification and send a slack message.

Prerequisite

To follow along with this, the reader should have basic knowledge and understanding of

Golang
AWS and its services ### Setting up the project First, we will start by writing out the function to send these notifications to Slack.

Create a new go project and call it whatever you want, I called mine “lambdaFunction” in your main.go file, paste the following piece of code



import (
   "bytes"
   "context"
   "encoding/json"
   "fmt"
   "github.com/aws/aws-lambda-go/events"
   "github.com/aws/aws-lambda-go/lambda"
   "net/http"
)

type slackMessage struct {
   Text string `json:"text"`
}

func handleRequest(ctx context.Context, snsEvent events.SNSEvent) error {
    webhookURL := "https://hooks.slack.com/services/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"
    for _, record := range snsEvent.Records {
        snsRecord := record.SNS
        sendSlackNotification(webhookURL, snsRecord.Message)
    }
    return nil
}

func sendSlackNotification(webhookURL, message string) {
   slackMessage := slackMessage{Text: "Cpu usage is above 50%" + message}
   slackBody, _ := json.Marshal(slackMessage)
   req, err := http.NewRequest(http.MethodPost, webhookURL, bytes.NewBuffer(slackBody))
   if err != nil {
      fmt.Printf("Error creating request: %v\n", err)
      return
   }
   req.Header.Set("Content-Type", "application/json")

   client := &http.Client{}
   resp, err := client.Do(req)
   if err != nil {
      fmt.Printf("Error sending request: %v\n", err)
   }
   defer resp.Body.Close()
   if resp.StatusCode != http.StatusOK {
      fmt.Printf("Error response from slack: %v\n", resp.StatusCode)
   } else {
      fmt.Printf("Successfully sent Slack notification: %v\n", resp.StatusCode)
   }
}



func main() {
   lambda.Start(handleRequest)
}

Run go mod

Let's try to understand what going on
The handleRequest function

First, we create a struct named slackMessage to represent the message format sent to Slack, it has a single field Text which holds the message content
The handleRequest function is the main function executed by the lambda runtime. It takes in two argument context and snsEvent events.SNSEvent containing details about the incoming SNS notification.
The function iterates through each snsRecord within the snsEvent, it retrieves the message content from the sns.message field and calls sendSlackNotification with the slack webhook URL and the message content

sendSlackNotification function

This function takes two arguments webhookURL: the URL where the Slack notifications are sent and message: the message content to be sent to Slack.
I provided a predefined message “cpu usage is above 50%” appended to the provided message. It then marshals the the slackMessage struct into JSON format using json.marshal.
An HTTP post request is created using http.NewRequest with the slack webhook URL, the JSON formatted body and the content type header set to application.json *The request is sent using an http.client and the response is received The main function is only used for local testing. In a lambda environment lamba.start function is automatically called with handleRequest as the entry point

Getting slack webhook URL

To obtain the Slack webhook URL that allows you to send messages to Slack, navigate to https://api.slack.com/apps. Make sure you are signed in to your Slack account before proceeding.

Click on “Create new app” on the top right side A dialog box will appear. Select "From scratch" to create a new app. Following that, another dialog box will pop up. Here, you can name your app "cpu-alert" and then choose the specific Slack workspace where you want the messages to be sent. I already created a test workspace “site reliability test”
Click “Create app”
In the “Add features and functionality” section select “Incoming webhooks”
Toggle the activate incoming webhook button to “on” Navigate back again and scroll to the “install app section”
Click "install to Workspace" then we will choose the channel we want Slack to send messages to. Then click allow.
Go back to “Add features and functionality” and select “Incoming webhooks”
Scroll down to find our webhook URL, then copy and paste it into our code.

The next step is to create a deployment package for our Go app
We will build the application.

Open a terminal in the project's working directory.run GOOS=linux go build -o main main.go
Create a ‘bootstrap’ file Create a file named ‘bootstrap’ in the project root directory with the following content ```

!/bin/sh

./main

Make the bootstrap file executable
* Run `chmod +x bootstrap`
* Zip the executable and the bootstrap file by running `zip function.zip main bootstrap`

Uploading the lamba function

* Navigate to the AWS management console
* Search for lambda, and create a new function
* Give it a name of your choice
* Select “Author from scratch”
* For the runtime, select Amazon linux 2023
* Click select function
* When the function is done creating scroll down and locate the “Upload from” option
* Select your function.zip file NOT the entire folder containing the code
* Save it
* Locate the runtime setting section and click on edit
* Set the handler to bootstrap and save it

In the next step, we'll configure a trigger for the Lambda function. This trigger defines the event that will prompt the function to send a message to Slack

As I mentioned earlier that trigger will be when cpu usage of a virtual machine is >= 50%

To achieve this functionality, the first step involves creating an EC2 instance.

When this is done we need to configure Cloudwatch to monitor and collect metrics

* Search for Cloudwatch and open it
* Select create alarms
* Choose Select metrics
* Select ec2
* Select per instance metrics
* Select CPUUtilization metric

In the condition section
* Select Greater/Equal for the threshold
* Define the threshold value as “50”
* Click next
On the next page locate the notification section
* We will leave the alarm state trigger as it is “In Alarm”
* Select the “create new topic” option
* Enter a unique name, you can also enter an email to receive notifications
* Select Create topic
* On the next page enter a unique alarm name
Then create alarm

We will head back to our lambda function
* Select “add trigger”
* In the “Select a source” field, Search for "sns" and select it
* Select the topic you created earlier and click "add"
### Testing
We’ve finished putting together the different parts of our simple infrastructure, now it's time to test.

To test that this works, we need to put our VM under a sort of stress test. This test generates a high CPU load. To perform this test we are going to be using the “stress” tool in linux.

First and foremost we need to install "stress" tool in our EC2 inatance.connect to the EC2 instance and Run the following commands
`sudo apt-get update`
`sudo apt-get install stress`
Use the following command to stress test your CPU
`stress --cpu 4 --timeout 300`
This example uses 4 CPU workers(number of parallel processes or threads) for 300 seconds(5 mins). You can adjust the number of workers and seconda as it suits you.

Open Slack and wait you should get an alert that looks like this

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/taas6tf5hvlp27qymsg3.png)
### Common Errors you might Encounter

While running your stress test, you might notice the state of Cloudwatch change to  “insufficient data” which might cause the alarm to delay for a bit. To fix this
* Open the Cloudwatch console
* Navigate to alarms and select your specific alarm
* Click on action then edit
* Scroll down to the missing data treatment section
* Select “Treat missing data as ignore(maintain the current state)”
* Save the alarm
### Conclusion
So far, we have explored how to write and set up a simple Lambda function in Go. Additionally, we’ve configured CloudWatch to monitor and collect metrics, set up CloudWatch alarms to trigger when specific thresholds are met, and established an SNS topic to receive these alarms. The purpose of the SNS topic is to trigger our Lambda function, which sends a Slack message.

I trust you found this enjoyable and informative. Should there be any errors or if any part was not explained clearly or you think I missed something, please feel free to reach out. Your feedback is highly valued. Thank You!

the link to the github repository is found [Here](https://github.com/audu97/lambdaFunction)

Jenkins Ci/Cd Pipeline to Build a Go Application into a Docker Image with Multistage build

Audu Ephraim — Sat, 29 Jun 2024 13:58:44 +0000

Introduction

In this article, i will be discussing how I implemented a ci/cd pipeline from scratch to build a simple Golang application into a docker image and push said image to Docker hub

The stages of the said pipeline include checking out the source code repository, in this case, git, running analysis on the source code with sonar cube to check for vulnerabilities, building the source code into a docker image using multi-stage build to reduce the size of the image, and finally pushing the image into a docker hub repository

Prerequisites:

Because the installation and configuration of these tools are long and a subject of another topic, I won't be discussing them here today.

The reader should have the following installed and configured and have basic knowledge and understanding of these tools if they wish to follow along.

Golang
Docker
IDE
Jenkins
Sonarcube
Git

The reader should also have the following accounts signed in

Docker hub
Sonarcloud
Jenkins
GitHub

Setting Up Project Files

The Application

As mentioned earlier I’ll be using a simple application written in Golang, this application has three different routes that print three different messages to the browser.

func firstEndPointHandler(w http.ResponseWriter, r *http.Request) {
   message := "this is the first endpoint"
   _, err := w.Write([]byte(message))
   if err != nil {
      log.Fatal(err)
   }
}


func secondEndPointHandler(w http.ResponseWriter, r *http.Request) {
   message := "second endpoint"
   _, err := w.Write([]byte(message))
   if err != nil {
      log.Fatal(err)
   }
}


func thirdEndPointHandler(w http.ResponseWriter, r *http.Request) {
   message := "second endpoint"
   _, err := w.Write([]byte(message))
   if err != nil {
      log.Fatal(err)
   }
}


func main() {
   http.HandleFunc("/first", firstEndPointHandler)
   http.HandleFunc("/second", secondEndPointHandler)
   http.HandleFunc("/third", thirdEndPointHandler)
   err := http.ListenAndServe(":8081", nil)
   log.Fatal(err)
}

Dockerfile

As you all know to build an application into a docker image I’ll need to use a docker file, where I'll specify the build for the image

In this docker file, I’ll be using a multistage build. With multistage builds, you can drastically reduce the size of a docker image and optimize the image by using multiple FROM statements, and each of them begins a new stage of a build. Each FROM statement can use a different base image.

The essence of this is to be able to copy only the necessary artefacts from one stage to another and leave the ones you don't need.

FROM golang:1.22.4 AS builder


WORKDIR /app
COPY go.mod ./
RUN go mod download


COPY *.go ./
RUN CGO_ENABLED=0 GOOS=linux go build -o /test-app


FROM scratch
COPY --from=builder /test-app /test-app


EXPOSE 8081


CMD ["/test-app"]

The first stage uses the golang:1.22.4 as the base image and it is named builder
Sets the working directory to /app
Copies go.mod file and download all dependencies
Copies all files that end with the .go extension
Builds the go binary with CGO_ENABLED=0 and GOOS=linux and produces an output image /test-app
The second stage of the build:
Uses the base image: scratch which is an empty image
Copies the built /test-app binary from the builder stage
Exposes port 8081
Specifies the command to run the binary: CMD[“/test-app”]

Jenkinsfile

The Jenkinsfile defines the pipeline stages and the steps executed during the pipeline. It must be placed in the root directory of the project for Jenkins to discover and initiate the pipeline.

pipeline {
   agent any


   tools {
      go 'golang'
   }
   environment {
       DOCKERHUB_CREDENTIALS = credentials('dockerhub')
       DOCKER_IMAGE = 'ephraimaudu/test-app'
       GITHUB_CREDENTIALS = 'git-secret'
       SONAR_TOKEN = credentials('SONAR_TOKEN')
   }


   stages{
       stage('Checkout'){
           steps{
               echo "checking out repo"
               git url: 'https://github.com/audu97/test-project', branch: 'master',
               credentialsId: "${GITHUB_CREDENTIALS}"
           }
       }
       stage('Run SonarQube Analysis') {
           steps {
               script {
                   echo 'starting analysis'
                   sh '/usr/local/sonar/bin/sonar-scanner -X -Dsonar.organization=eph-test-app -Dsonar.projectKey=eph-test-app-test-go-app -Dsonar.sources=. -Dsonar.host.url=https://sonarcloud.io'
               }
           }
       }
       stage('Run Docker Build'){
           steps{
               script{
                    echo "starting docker build"
                    sh "docker build build -t ${DOCKER_IMAGE}:${env.BUILD_ID} ."
                    echo "docker built successfully"
               }
           }
       }
       stage('push to docker hub'){
           steps{
               echo "pushing to docker hub"
               script{
                   docker.withRegistry('https://index.docker.io/v1/', 'dockerhub'){
                       docker.image("${DOCKER_IMAGE}:${env.BUILD_ID}").push()
                   }
               }
               echo "done"
           }
       }
   }


   post {
       always{
           cleanWs()
       }
   }
}

NOTE: To use these credentials in the environment variables, you should add them in the “Credentials” section within the “Manage Jenkins” option of the Jenkins UI. This way, your Jenkins jobs can securely access the necessary credentials during their execution.
The agent specifies that the pipeline can run on any available executor in the Jenkins environment
Environment variables: defines various environment variables used in the pipeline execution. DOCKERHUB_CREDENTIALS -contains credentials for signing in to docker hub, GITHUB_CREDENTIALS -contains credentials to use to sign in to GitHub to check out the specified repository, SONAR_TOKEN- also serves as credentials for the Sonar cloud, where I can view the code analysis, DOCKER_IMAGE-specifies the name I want for the docker image.
Stages: the pipeline consists of several stages:

Checkout: This stage checks out the code from the specified GitHub repository
Run sonarqube analysis: executes sonar cube analysis on the code base. Sonar cube is a tool for static analysis on a code base by analyzing it statically. It detects bugs, vulnerabilities and code smells.
Run docker build: builds a docker image using the specified dockerfile.
Push to docker hub: pushes the built docker image to docker hub
Post processing: the post section ensures that the workspace is cleaned up after the pipeline execution, even if the pipeline fails

Challenges

The biggest challenge I faced, which took considerable time to resolve, was that Jenkins could not locate Docker to execute the Docker build stage in my pipeline because I had installed both Jenkins and Docker using snaps. This resulted in repeated pipeline failures.
To overcome this issue, I uninstalled the snap versions of both Jenkins and Docker. Following that, I installed them following the instructions provided in their documentation. This approach solved my problem by allowing Jenkins to interact with Docker successfully.

Conclusion

This project has provided valuable insights into the importance and the need for CI/CD pipelines. Additionally, it emphasizes multistage Docker image builds and includes security considerations during the build stage (shifting security left) by using SonarQube.

The link to the repository containing the entire project can be located HERE

Deploying a Kubernetes Cluster on Azure Kubernetes Service(AKS) with Terraform

Audu Ephraim — Sun, 16 Jun 2024 15:43:15 +0000

Introduction

Kubernetes, often abbreviated as K8s, is an open-source platform designed to automate deploying, scaling, and operating application containers. It allows you to manage containerized applications across a cluster of machines efficiently

Azure Kubernetes Service(AKS): Is a managed Kubernetes service offered by Microsoft as part of the Azure cloud platform. It provides a way for organizations to deploy and manage their containerized applications at scale, leveraging the powerful features of Kubernetes.

By providing a fully managed service that handles many of the underlying infrastructure and management chores, AKS makes it easier to build and operate Kubernetes clusters.

Because of this, businesses can concentrate on their apps and services rather than worrying about the infrastructure as a whole.

Terraform is an open-source infrastructure-as-code software tool created by HashiCorp. It allows users to define and provision their infrastructure on different cloud platforms and define services using a high-level configuration language known as HashiCorp Configuration Language (HCL), or optionally JSON.

In this article, I will be discussing how I created an AKS cluster on Azure entirely using Terraform, provisioned an NGINX image and set up Prometheus and Grafana for monitoring and alerting.

This article assumes the reader has a basic understanding of Azure and Kubernetes and also has the Azure CLI installed and signed in.

Creating the cluster

To begin, I created a directory named azure-aks to store my Terraform scripts. Then, I created a main.tf file where I will write the Terraform configurations for my resources.

In the main.tf file, I began by defining the providers that I will need for this project.

Terraform Providers are plugins that implement resource types and data sources. They serve as a bridge between Terraform and a service or platform, such as Azure, AWS, Kubernetes, etc.

In my case, I will need providers for Azure to communicate with Azure services, a provider for Kubernetes to create Kubernetes services, and Helm to access Helm charts for Prometheus and Grafana.


terraform {
 required_providers {
   azurerm = {
     source = "hashicorp/azurerm"
     version = "3.107.0"
   }


   kubernetes = {
     source = "hashicorp/kubernetes"
     version = "2.30.0"
   }


   helm = {
     source = "hashicorp/helm"
     version = "2.13.2"
   }
 }
}

After that i ran terraform init so terraform downloads the plugin and the neccessary dependencies needed.
Next creating resource groups and the Kubernetes cluster

resource "azurerm_resource_group" "aks-resource" {
 name     = "aks-resources"
 location = "France Central"
}


resource "azurerm_kubernetes_cluster" "test_cluster" {
 name                = "example-aks1"
 location            = azurerm_resource_group.aks-resource.location
 resource_group_name = azurerm_resource_group.aks-resource.name
 dns_prefix          = "testaks"


 default_node_pool {
   name       = "default"
   node_count = 2
   vm_size    = "Standard_D2_v2"
 }


 identity {
   type = "SystemAssigned"
 }


 tags = {
   Environment = "Production"
 }
}

Created a resource group with the name aks-resource in the “France Central” region and an Azure Kubernetes Service(AKS) named “example-aks1” within that resource group. The AKS cluster will have a default node pool with 2(which can be increased to suit your need) nodes of size “Standard_D2_v2” and it will use a system-assigned managed identity. The cluster is also tagged “Environment: Production”.

Once this is done, I ran the terraform plan command to see the resources that will be provisioned then terraform apply to provision these resources.

The reason I did this was, in other to get the kubeconfig file I have to run the command
az aks get-credentials --resource-group <ResourceGroupName> --name <AKSClusterName> (replaced ResourceGroupName and AKSClusterName with the name of my resource group and my cluster name)

The kubeconfig file is used to allow access to the Kubernetes clusters. It contains the necessary details to connect to the cluster, such as cluster API server addresses, user credentials, and namespaces.

This file is used by Kubectl and other Kubernetes client applications to communicate with the cluster’s API server and manage Kubernetes resources.

Essentially, it’s like a key that allows you to access and control your Kubernetes cluster on the cloud.

After running the az aks get-credentials --resource-group <ResourceGroupName> --name <AKSClusterName> command the terminal outputs where the kubeconfig file is saved.

data "azurerm_kubernetes_cluster" "test_cluster" {
 name = azurerm_kubernetes_cluster.test_cluster.name
 resource_group_name = azurerm_resource_group.aks-resource.name
}

The data block is used to fetch data about an existing aks cluster. It retrieves information about the aks cluster with the specified name and resource group, which is to be used somewhere else in the terraform configuration

resource "local_file" "kubeconfig" {
 content = data.azurerm_kubernetes_cluster.test_cluster.kube_config_raw
 filename = "/home/ephraim/.kube/config"
 }

The resource "local_file" "kubeconfig" block creates a local file that contains the kubeconfig of the retrieved AKS cluster. This kubeconfig is necessary to interact with my Kubernetes cluster using kubectl or other Kubernetes tools.

The content of the file is the raw kubeconfig data from the AKS cluster, and it’s being saved to a specified path on my local machine (/home/ephraim/.kube/config)

resource "null_resource" "wait_for_kubeconfig" {
 provisioner "local-exec" {
   command = "sleep 10"


 }


 depends_on = [ local_file.kubeconfig ]
 }


provider "kubernetes" {
 config_path = local_file.kubeconfig.filename
}


provider "helm" {
   kubernetes {
     config_path = local_file.kubeconfig.filename
   }
}

The null resource block is used to introduce a delay in the terraform execution, which pauses the execution for 10 seconds to ensure that the kubeconfig file is fully written.
The provider "kubernetes" block configures the Kubernetes provider for Terraform, which allows me to manage my Kubernetes resources with Terraform. It uses the kubeconfig file created by the local_file.kubeconfig resource to connect to my AKS cluster.
Similarly, the provider "helm" block configures the Helm provider, which lets me deploy Helm charts to my Kubernetes cluster. It also uses the same kubeconfig file for connectivity.

resource "kubernetes_namespace" "test_namespace" {
 metadata{
   name = "monitoring"
 }


 depends_on = [ local_file.kubeconfig ]
}

Resource “kubernetes namespace” creates a kubernetes namespace called “monitoring”

A Kubernetes namespace is a way to divide cluster resources between multiple users. It is some sort of cluster within the Kubernetes cluster but for similar tasks or similar resources. Or let me say, it is used for grouping similar cluster resources to bolster organisation.
In this namespace, I’m going to be provisioning Prometheus and Grafana.
The “depends_on” attribute ensures that the namespace is not created until the local_file.kubeconfig file is applied. This means that Terraform will wait for the kubeconfig file to be available before it attempts to create the namespace

resource "helm_release" "prom-helm" {
   name = "prometheus"
   repository = "https://prometheus-community.github.io/helm-charts"
   chart      = "prometheus"
   namespace  = kubernetes_namespace.test_namespace.metadata[0].name
   depends_on = [ kubernetes_namespace.test_namespace ]
}


resource "helm_release" "graf-helm" {
   name = "grafana"
   repository = "https://grafana.github.io/helm-charts"
   chart      = "grafana"
   namespace  = kubernetes_namespace.test_namespace.metadata[0].name
   depends_on = [ kubernetes_namespace.test_namespace ]
 }

The resource "helm_release" "prom-helm" block will deploy Prometheus from the specified Helm chart repository. It sets the release name to Prometheus, uses the chart from the Prometheus community Helm repository, and deploys it to the monitoring namespace created by the kubernetes_namespace.test_namespace resource.

The resource "helm_release" "graf-helm" block does the same for Grafana, deploying it from the Grafana Helm chart repository with the release name Grafana to the same monitoring namespace.
Both resources have a depends_on attribute that ensures they are created after the monitoring namespace has been created. Since they are both being deployed in the monitoring namespace.

resource "kubernetes_deployment" "nginx_depl" {
   metadata {
     name = "nginx-deployment"
     namespace = kubernetes_namespace.test_namespace.metadata[0].name
   }
   spec {
     replicas = 2


     selector {
       match_labels = {
           app = "nginx"
       }
     }


     template {
       metadata {
         labels = {
           app = "nginx"
         }
       }


       spec {
         container {
           name = "nginx"
           image = "nginx:latest"


           port {
             container_port = 80
           }
         }
       }
     }
   }
   depends_on = [ kubernetes_namespace.test_namespace ]
}

This defines a Kubernetes deployment named “nginx-deployment” that will be created in the monitoring namespace. This deployment will set up two NGINX pods running in my Kubernetes cluster within the monitoring namespace serving content on port 80.
The depends_on attribute also makes sure it is created only after the namespace has been created

resource "kubernetes_service" "nginx-service" {
   metadata {
     name = "nginx-service"
     namespace = kubernetes_namespace.test_namespace.metadata[0].name
   }


   spec {
     selector = {
       app = "nginx"
     }


     port {
       port = 80
       target_port = 80
     }


     type = "LoadBalancer"
   }
   depends_on = [ kubernetes_namespace.test_namespace]
}

This resource creates a Kubernetes service called “nginx-service” also in the monitoring namespace of type LoadBalancer. This also depends on the namespace created earlier
the full code looks like this

terraform {
  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
      version = "3.107.0"
    }

    kubernetes = {
      source = "hashicorp/kubernetes"
      version = "2.30.0"
    }

    helm = {
      source = "hashicorp/helm"
      version = "2.13.2"
    }
  }
}

provider "azurerm" {
  # Configuration options
  features {}
}


resource "azurerm_resource_group" "aks-resource" {
  name     = "aks-resources"
  location = "France Central"
}

resource "azurerm_kubernetes_cluster" "test_cluster" {
  name                = "example-aks1"
  location            = azurerm_resource_group.aks-resource.location
  resource_group_name = azurerm_resource_group.aks-resource.name
  dns_prefix          = "testaks"

  default_node_pool {
    name       = "default"
    node_count = 2
    vm_size    = "Standard_D2_v2"
  }


  identity {
    type = "SystemAssigned"
  }

  tags = {
    Environment = "Production"
  }
}

data "azurerm_kubernetes_cluster" "test_cluster" {
  name = azurerm_kubernetes_cluster.test_cluster.name
  resource_group_name = azurerm_resource_group.aks-resource.name
}

resource "local_file" "kubeconfig" {
  content = data.azurerm_kubernetes_cluster.test_cluster.kube_config_raw
  filename = "/home/ephraim/.kube/config"

}

resource "null_resource" "wait_for_kubeconfig" {
  provisioner "local-exec" {
    command = "sleep 10"

  }

  depends_on = [ local_file.kubeconfig ]

}

provider "kubernetes" {
  config_path = local_file.kubeconfig.filename
}

provider "helm" {
    kubernetes {
      config_path = local_file.kubeconfig.filename
    }
}


resource "kubernetes_namespace" "test_namespace" {
  metadata{
    name = "monitoring"
  }

  depends_on = [ local_file.kubeconfig ]
}


resource "helm_release" "prom-helm" {
    name = "prometheus"
    repository = "https://prometheus-community.github.io/helm-charts"
    chart      = "prometheus"
    namespace  = kubernetes_namespace.test_namespace.metadata[0].name
    depends_on = [ kubernetes_namespace.test_namespace ]
}

resource "helm_release" "graf-helm" {
    name = "grafana"
    repository = "https://grafana.github.io/helm-charts"
    chart      = "grafana"
    namespace  = kubernetes_namespace.test_namespace.metadata[0].name
    depends_on = [ kubernetes_namespace.test_namespace ]

}

resource "kubernetes_deployment" "nginx_depl" {
    metadata {
      name = "nginx-deployment"
      namespace = kubernetes_namespace.test_namespace.metadata[0].name
    }
    spec {
      replicas = 2

      selector {
        match_labels = {
            app = "nginx"
        }
      }

      template {
        metadata {
          labels = {
            app = "nginx"
          }
        }

        spec {
          container {
            name = "nginx"
            image = "nginx:latest"

            port {
              container_port = 80
            }
          }
        }
      }
    }
    depends_on = [ kubernetes_namespace.test_namespace ]
}


resource "kubernetes_service" "nginx-service" {
    metadata {
      name = "nginx-service"
      namespace = kubernetes_namespace.test_namespace.metadata[0].name
    }

    spec {
      selector = {
        app = "nginx"
      }

      port {
        port = 80
        target_port = 80
      }

      type = "LoadBalancer"
    }
    depends_on = [ kubernetes_namespace.test_namespace]
}

output "client_certificate" {
  value     = azurerm_kubernetes_cluster.test_cluster.kube_config[0].client_certificate
  sensitive = true
}

output "kube_config" {
  value = azurerm_kubernetes_cluster.test_cluster.kube_config_raw

  sensitive = true
}

Deployment

To deploy the finalized infrastructure to Azure, I will need to run terraform plan to preview the resources that will be created. Following this, I'll execute terraform apply to provision these resources. This process may take some time.

Verify Deployment

To verify the successful deployment, I navigated to the ‘Connect’ tab in the cluster portal, where Azure provides commands for authentication and connection to my cluster. After executing these commands, I successfully connected to my cluster. To view all my deployments, I ran the command kubectl get deployments --namespace monitoring

Everything seems to be up and running correctly

Additionally, I needed to verify if the Nginx service was set up correctly. Once Nginx, of type LoadBalancer, had been deployed, Kubernetes provisioned an external IP. To access it, I ran the command kubectl get svc nginx-service --namespace monitoring. In the ‘External IP’ column, I copied and pasted the IP address into a browser. Nginx is running correctly!

Ran “terraform destroy” to delete and remove all resources

Challenges

The major challenge I faced was obtaining and using the kubeconfig file. I later realized that I needed to create the resource group and cluster first, then retrieve the kubeconfig file, before proceeding to create the other resources.

Conclusion

In this article, I’ve walked through the process of deploying a Kubernetes cluster on Azure Kubernetes Service (AKS) using Terraform. I deployed an AKS cluster and configured Kubernetes resources, including Nginx, Prometheus, and Grafana for my application’s needs

To take this further I intend to explore topics such as auto-scaling, continuous deployment pipelines, and multi-region clusters to further enhance your Kubernetes infrastructure.

the github repo for the full code can be found Here

Azure Resume Challenge Using Pulumi, Golang, and Azure Blob Storage

Audu Ephraim — Thu, 06 Jun 2024 12:56:03 +0000

I undertook the resume challenge on Azure. More about the resume challenge can be found in the AWS resume challenge which can be found Here.

The whole process was almost the same as the AWS challenge except for a few differences, which I will be pointing out.

Setting up the environment

Setting up the environment remained consistent except for a few changes when starting the project, I used “pulumi new azure-go” for an Azure project

Creating the Infrastructure

Because in Azure, every resource belongs to a resource group and blob storages belong to storage groups, I had to create a resource group and a storage group, also I had to create a static website on an Azure storage account.

Since Azure Blob Storage is an object store similar to AWS S3, I need to handle each file in my website folder, which consists of JavaScript, jQuery, and CSS code, as an object. To do this, I'll need to traverse the folder and create a corresponding blob for each file.

I created a new file blobFolder.go and added the following

type BlobFolder struct {
   pulumi.ResourceState

   containerName pulumi.StringOutput `pulumi:"containerName"`
}

func NewBlobFolder(ctx *pulumi.Context, containerName string, siteDir string, args *FolderArgs) (*BlobFolder, error) {

   var resource BlobFolder
   // Stack exports
   err := ctx.RegisterComponentResource("pulumi:example:BlobFolder", containerName, &resource)
   if err != nil {
      return nil, err
   }
   //creating a resource group
   resourceGroup, err := resources.NewResourceGroup(ctx, "new-resource-group", nil)
   if err != nil {
      return nil, err
   }
   //creating an azure storage account
   account, err := storage.NewStorageAccount(ctx, "newstorage", &storage.StorageAccountArgs{
      ResourceGroupName: resourceGroup.Name,
      Sku: &storage.SkuArgs{
         Name: pulumi.String("Standard_LRS"),
      },
      Kind: pulumi.String("StorageV2"),
   })
   if err != nil {
      return nil, err
   }
   //creating a static website
   staticWebsite, err := storage.NewStorageAccountStaticWebsite(ctx, "staticWebsite", &storage.StorageAccountStaticWebsiteArgs{
      AccountName:       account.Name,
      ResourceGroupName: resourceGroup.Name,
      IndexDocument:     pulumi.String("index.html"),
   })
   if err != nil {
      return nil, err
   }

   // For each file in the directory, create a blob object
   err = filepath.Walk(siteDir, func(name string, info fs.FileInfo, err error) error {
      if err != nil {
         return err
      }
      if !info.IsDir() {
         rel, err := filepath.Rel(siteDir, name)
         if err != nil {
            return err
         }

         _, err = storage.NewBlob(ctx, rel, &storage.BlobArgs{
            ResourceGroupName: resourceGroup.Name,
            AccountName:       account.Name,
            ContainerName:     staticWebsite.ContainerName,
            Source:            pulumi.NewFileAsset(name),
            ContentType:       pulumi.String(mime.TypeByExtension(path.Ext(name))),
         })
         if err != nil {
            return err
         }
      }
      return nil
   })
   if err != nil {
      return nil, err
   }

   ctx.Export("staticWebsite", account.PrimaryEndpoints)

   return &resource, nil
}

type folderArgs struct {
}

type FolderArgs struct {
}

func (FolderArgs) ElementType() reflect.Type {

   return reflect.TypeOf((*folderArgs)(nil)).Elem()
}

then in main.go

func main() {
    pulumi.Run(func(ctx *pulumi.Context) error {

        f, err := NewBlobFolder(ctx, "resume-container", "./website", &FolderArgs{})
        if err != nil {
        }
        ctx.Export("bucketName", f.containerName)
        return nil
    })
}

Conclusion

This project provided valuable insights into how both cloud platforms, Azure and AWS, are different yet very similar. Over time, I plan to extend this project and explore using different infrastructures.

link to the github repo https://github.com/audu97/azure-resume-challenge.git

AWS Resume Challenge using Pulumi, Golang, AWS S3 and AWS CloudFront

Audu Ephraim — Mon, 03 Jun 2024 01:07:34 +0000

1. Introduction

I recently undertook the AWS resume challenge, which was quite interesting, so I decided to write about it.

The primary goal of the AWS resume challenge is for participants to build a static personal website using cloud services.

it encourages hands-on experience with various cloud technologies including infrastructure as code(IaC), serverless computing and storage devices.

For this challenge, I utilised the following:

Amazon Web Service(AWS): Served as my cloud provider.
AWS S3: Acted as a storage unit to host my static files.
AWS Cloudfront: as a content delivery network, to serve the static webpage. A content delivery network(CDN) is a geographically distributed group of servers that caches content close to users.
Pulumi: Used as my infrastructure as code(IaC) tool. Pulumi is an IaC that lets you describe and provision your cloud infrastructure using various programming languages.

2. Setting Up the Environment

The first steps were to install the necessary tools and software including Golang, an IDE(Goland) or any IDE of your choice and pulumi.
Next was to configure pulumi to work with my AWS account using the AWS CLI
Made sure I had the necessary permission to create and manage AWS resources in my AWS account.
Created a directory to initialise the Pulumi project. To do this I opened my terminal and ran the following command mkdir aws-resume-challenge && cd aws-resume-challenge This creates a new folder and navigates into it.
Then I ran pulumi new aws-go This creates a new pulumi aws project and downloads the go sdk. This also generates some files that are necessary for the program to execute successfully.

3.Creating the infrastructure

I copied my static website folder which contains some HTML, CSS, JavaScript, and jQuery code and my index.html file into the root of my project.

Because I have a folder containing about 6 different folders with different files I need an object for each file(each file is a separate resource). I need a program to crawl the directory and add a resource (Bucket object) for each file. I’ll use a component that will handle the folder, to do this I created another go file and called it s3folder.go which contains the following code:

type Folder struct {
   pulumi.ResourceState

   bucketName pulumi.IDOutput     `pulumi:"bucketName"`
   websiteUrl pulumi.StringOutput `pulumi:"websiteUrl"`
}

func NewS3Folder(ctx *pulumi.Context, bucketName string, siteDir string, args *FolderArgs) (*Folder, error) {

   var resource Folder
   // Stack exports
   err := ctx.RegisterComponentResource("pulumi:example:S3Folder", bucketName, &resource)
   if err != nil {
      return nil, err
   }
   // Create a bucket and expose a website index document
   siteBucket, err := s3.NewBucket(ctx, bucketName, &s3.BucketArgs{
      Website: s3.BucketWebsiteArgs{
         IndexDocument: pulumi.String("index.html"),
      },
   }, pulumi.Parent(&resource))
   if err != nil {
      return nil, err
   }

   // For each file in the directory, create an S3 object stored in `siteBucket`
   err = filepath.Walk(siteDir, func(name string, info fs.FileInfo, err error) error {
      if err != nil {
         return err
      }
      if !info.IsDir() {
         rel, err := filepath.Rel(siteDir, name)
         if err != nil {
            return err
         }

         if _, err := s3.NewBucketObject(ctx, rel, &s3.BucketObjectArgs{
            Bucket:      siteBucket.ID(),                                     
            Source:      pulumi.NewFileAsset(name),                           
            ContentType: pulumi.String(mime.TypeByExtension(path.Ext(name))), 
         }, pulumi.Parent(&resource)); err != nil {
            return err
         }
      }
      return nil
   })
   if err != nil {
      return nil, err
   }

   return &resource, nil
}

type folderArgs struct {
}

type FolderArgs struct {
}

func (FolderArgs) ElementType() reflect.Type {

   return reflect.TypeOf((*folderArgs)(nil)).Elem()
}

FolderStruct: is a custom pulumi resource that has two properties, bucket name and website name. These are populated after the resource is created.
newS3folder function: this function creates a new instance of the folder resource. The function takes in a pulumi context, a bucket name, a directory path and arguments for the folder resource. It first creates a new component resource of type pulumi:example:s3folder The s3 bucket is created with a website configuration that sets the index.html file as the index of the document It goes through the provided directory for each file, it creates a new s3 bucket object. BucketobjectArgs specifies the bucket, using the bucket id, it also specifies the source file and the content type of the file which is determined by the file extension. In main.go:

func main() {
   pulumi.Run(func(ctx *pulumi.Context) error {
      // Create a bucket and expose a website index document
      f, err := NewS3Folder(ctx, "resume-bucket", "./website", &FolderArgs{})
      if err != nil {
         return err
      }
      ctx.Export("bucketName", f.bucketName)
      ctx.Export("websiteUrl", f.websiteUrl)
      return nil
   })
}

This is the entry point of a pulumi program, using the pulumi.Run func. This is where the infrastructure is defined. Firstly
It calls the newS3Folder function to create a new bucker called “resume bucket” and uploads the contents of the /website directory to this bucket. It returns a folder resource
When the folder resource is created successfully it exports two outputs bucketname and websiteurl in the pulumi CLI

To deploy this infrastructure, I ran pulumi up command which creates these resources.

4. Configuring AWS Cloudfront for the s3Bucket:

I created a new Cloudfront distribution to properly serve this web page with low latency

Chose the resume bucket as the origin domain
Under origin access, I selected Origin access control settings and created a new AoC, which will be provided when the distribution is created
Under the viewer protocol policy, I selected “https only” which means CloudFront will only accept HTTPS requests Created the distribution and copied the policy
Navigated to the bucket, then to the permission tab I edited the bucket policy, which says only traffic from Cloudfront should be allowed

5. Conclusion

In conclusion, participating in the AWS resume challenge has been rewarding, it allowed me to use my skills in Golang especially in crawling through the website folder and creating a bucket object for every one of the files, also My AWS skills were sharpened.

I intend to extend this and add further features to this static website like ci/cd. To automatically update the website when I make and push changes to it. Bye for now!🤗

the github link to the full project can be found here

Efficiently Deploy Cloud Resources on AWS with Terraform

Audu Ephraim — Thu, 02 May 2024 23:52:46 +0000

Infrastructure as code(IaC) tools has become very popular and useful in cloud engineering. Tools like Chef, puppet, and Ansible are configuration management tools, focusing on installing and managing applications on existing servers.

Provisioning tools like Pulumi, Crossplane and Terraform create new servers and other infrastructure components.

Terraform is a declarative coding tool that uses a high-level configuration language called Hashicorp configuration language(HCL) to describe the desired cloud or on-premise infrastructure for running an application.

In this article, we will build a simple cloud architecture with a virtual private cloud(VPC), subnets, route tables, security groups and an ec2 instance and deploy a web server all using a Terraform script.

Prerequisites

Basic knowledge of cloud environments(AWS)
An AWS account
Terraform. You can download it from here
VS Code with terraform plugin. VS code can be downloaded from its official website. Once installed navigate to the plugin marketplace and search for “terraform” and install.

Understanding the infrastructure

VPC: Virtual Private Cloud is an isolated section of the AWS cloud where you can launch AWS resources in a defined virtual network. It provides isolation and control over our network environment which includes IP address range selection, subnet creation and route table configurations.
Subnets: Subnets are a range of IP addresses in our VPC that you designate to group resources based on security requirements.
Route Table: is a set of rules, known as routes that are used to determine where traffic is directed within a VPC or to an external network
Internet Gateway: An Internet gateway enables communications between instances within a VPC and the Internet. It allows them to send data to and from the internet.
Security groups: Security groups act as virtual firewalls for your server instances. It controls inbound and outbound traffic. It allows us to define which traffic is allowed to reach our instances.
Elastic IP: Elastic IP is a static IPv4 address assigned to your instance. When an AWS instance is restarted, the IP address changes. An elastic IP enables us to attach a persistent IP address to our instance.
EC2 Instance: An EC2 instance is a virtual server in AWS that hosts our application.

Setting Up Terraform configuration

To begin writing our terraform script:

Create a folder, lets call it "terraform-project"
Open this folder in vs code and create a file within this folder, we'll call it main.tf for the sake of this guide.

Let's write some scripts.

AWS Provider
The first step in any Terraform script is to define the provider. In our case, we're using AWS.

The provider block is used to specify the details of the cloud provider, such as Azure, GCP, Oracle Cloud, and even container orchestration platforms like Kubernetes, where all the resources will be created. Here's what our provider block would look like:

provider "aws" {
  region     = "us-east-1"
  access_key = ""
  secret_key = ""
}

provider "aws" - This line declares a provider block for AWS. Terraform uses a plugin-based architecture and “aws” is the plugin for Amazon Web Service.
region = "us-east-1" - This line sets the AWS region where the resources will be created. In this case, it’s set to “us-east-1”, which corresponds to the N. Virginia region.
access_key = “” - this line is where you should put your AWS access key. It is part of your AWS security credentials
Secret_key = “”- this line is where you should put your AWS access key.
To obtain your secret and access key, follow these steps:

Navigate to the AWS Management Console.
Click on your name in the top right corner and select “Security Credentials”.

Select “Create Access Key”. Your access and secret key will be provided.

Don’t forget to download the CSV file for safekeeping. Remember, these keys are sensitive information. Store them securely and do not share them publicly.

Virtual Private Cloud(VPC)

resource "aws_vpc" "prod-vpc" {
  cidr_block = "10.0.0.0/16"
  tags = {
    Name = "production"
  }
}

resource ”aws vpc” “prod-vpc”- this line declares a resource of type “aws vpc” (an AWS VPC) and gives it the name "prod-vpc". This name is used to refer to this resource in other parts of our terraform script
Cidr_block = “10.0.0.0/16” - This line sets the IP address range for the VPC in CIDR notation. The range “10.0.0.0/16” includes all IP addresses from 10.0.0.0 to 10.0.255.255.
Tags{Name = production} - Tags are key-value pairs that you can attach to AWS resources to easily organize, search, and identify them. In our case, our VPC has the tag “production”.

Internet Gateway

resource "aws_internet_gateway" "gw" {
    vpc_id = aws_vpc.prod-vpc.id
}

resource “aws_internet_gateway” “gw” - This line declares a resource of type “aws_internet_gateway” (an AWS Internet Gateway), and gives it the name “gw”. This name is used to refer to this resource in other parts of our Terraform script
vpc_id = aws_vpc.prod-vpc.id - This line attaches the Internet Gateway to the VPC that was created earlier in the script. It does this by setting the vpc_id property of the Internet Gateway to the ID of the VPC. The aws_vpc.prod-vpc.id syntax is used to get the ID of the VPC named prod-vpc.

Route Table

resource "aws_route_table" "prod-route-table" {
 vpc_id = aws_vpc.prod-vpc.id
 route {
   cidr_block = "0.0.0.0/0"
   gateway_id = aws_internet_gateway.gw.id
 }
  tags = {
   Name = "prod"
 }
}

resource “aws_route_table” “prod-route-table” - This line is declaring a resource of type “aws_route_table” and the name of this particular resource is “prod-route-table”.
vpc_id = aws_vpc.prod_vpc.id - this line associates the route table with the VPC that has been created earlier in the script with the name prod-vpc. The .id attribute fetches the ID of that VPC.
The route block within the resource defines a route for the route table.
cidr_block = "0.0.0.0/0"- This line specifies the destination for the route as 0.0.0.0/0, which represents all IPv4 addresses. In the context of a route table, this is often used to define the default route.
gateway_id = aws_internet_gateway.gw.id - This line is setting the Internet Gateway (gw which was declared earlier) as the target of the route, meaning that any traffic destined for 0.0.0.0/0 will be directed to this Internet Gateway.

Subnet

resource "aws_subnet" "subnet-1" {
   vpc_id = aws_vpc.prod-vpc.id
   cidr_block = "10.0.1.0/24"
   availability_zone = "us-east-1a"
   tags = {
       Name = "prod"
   }
}

resource “aws_subnet” “subnet-1” - This line declares a resource of type aws_subnet and the name of this particular resource is subnet-1.
vpc_id = aws_vpc.prod-vpc.id - This line associates the subnet with the VPC that has been created earlier in the script with the name prod-vpc. The .id attribute fetches the ID of that VPC.
cidr_block = "10.0.1.0/24" - This line specifies the IP address range for the subnet in CIDR notation. The range 10.0.1.0/24 includes all IP addresses from 10.0.1.0 to 10.0.1.255.
availability_zone = "us-east-1a" - This line specifies the availability zone in which to create the subnet. In this case, it’s us-east-1a.

Route Table Association

resource "aws_route_table_association" "a" {
   subnet_id = aws_subnet.subnet-1.id
   route_table_id = aws_route_table.prod-route-table.id
}

resource “aws_route_table_association” ”a” - This line is declaring a resource of type aws_route_table_association and the name of this particular resource is a.
subnet_id = aws_subnet.subnet-1.id - This line associates the route table with the subnet that has been created earlier in the script with the name subnet-1. The .id attribute fetches the ID of that subnet.
route_table_id = aws_route_table.prod-route-table.id This line specifies the route table to associate with the subnet. It refers to the route table that was created earlier with the name prod-route-table.
This block means that the routing policies defined in the prod-route-table will now apply to any resources that are deployed within subnet-1

Security Group

resource "aws_security_group" "allow-web" {
    name = "allow_web_traffic"
    description = "allow traffic"
    vpc_id = aws_vpc.prod-vpc.id

    ingress {
        description = "https"
        from_port        = 443
        to_port          = 443
        protocol         = "tcp"
        cidr_blocks      = ["0.0.0.0/0"]
    }

    ingress {
        description = "http"
        from_port        = 80
        to_port          = 80 
        protocol         = "tcp"
        cidr_blocks      = ["0.0.0.0/0"]
    }

    ingress {
        description = "ssh"
        from_port        = 22
        to_port          = 22
        protocol         = "tcp"
        cidr_blocks      = ["0.0.0.0/0"]
    }

    tags = {
      Name = "allow web"
    }
}

resource “aws_security_group” “allow-web” - This line declares a resource of type aws_security_group and the name of this particular resource is "allow-web".
name = "allow_web_traffic" This line is setting the security group's name to "allow_web_traffic".
description = "allow traffic" - This line sets the description of the security group to allow traffic.
vpc_id = aws_vpc.prod-vpc.id - This line associates the security group with the VPC that has been created earlier in the script with the name "prod-vpc". The .id attribute fetches the ID of that VPC.

The ingress blocks within the resource define inbound rules for the security group

The first ingress block allows inbound HTTPS traffic on port 443 from any IP address (0.0.0.0/0).
The second ingress block allows inbound HTTP traffic on port 80 from any IP address (0.0.0.0/0).
The third ingress block allows inbound SSH traffic on port 22 from any IP address (0.0.0.0/0). ** Network Interface**

resource "aws_network_interface" "web-server-nic" {
  subnet_id       = aws_subnet.subnet-1.id
  private_ips     = ["10.0.1.50"]
  security_groups = [aws_security_group.allow-web.id]
}

resource "aws_network_interface" "web-server-nic" - This line declares a resource of type aws_network_interface and the name of this particular resource is "web-server-nic".
subnet_id = aws_subnet.subnet-1.id - This line associates the network interface with the subnet that has been created earlier in the script with the name "subnet-1." The .id attribute fetches the ID of that subnet.
private_ips = ["10.0.1.50"] - This line assigns the private IP address 10.0.1.50 to the network interface.
security_groups = [aws_security_group.allow-web.id] This line associates the network interface with the security group that has been created earlier in the script with the name "allow-web" The .id attribute fetches the ID of that security group.

Elastic IP

resource "aws_eip" "eip" {
    network_interface = aws_network_interface.web-server nic.id
    associate_with_private_ip = "10.0.1.50"
    depends_on = [ aws_internet_gateway.gw ]
}

resource "aws_eip" "eip" - This line declares a resource of type aws_eip and the name of this particular resource is true.
network_interface = aws_network_interface.web-server-nic.id - This line associates the EIP with the network interface that has been created earlier in the script with the name "web-server-nic". The .id attribute fetches the ID of that network interface.
associate_with_private_ip = "10.0.1.50" - This line specifies the private IP address with which the EIP should be associated. In this case, it’s 10.0.1.50, which is the same private IP address assigned to the network interface.
depends_on = [ aws_internet_gateway.gw ] This line specifies a dependency on the Internet Gateway resource named "gw". This means that Terraform will ensure the Internet Gateway is created before attempting to create the EIP.
EC2 Instance

resource "aws_instance" "web-server" {
    ami = "ami-04b70fa74e45c3917"
    instance_type = "t2.micro"
    availability_zone = "us-east-1a"
    key_name = "main-key2"

    network_interface {
        device_index = 0
        network_interface_id = aws_network_interface.web-server-nic.id

    }

    user_data = <<-EOF
      #!/bin/bash
      sudo apt update -y
      sudo apt install apache2 -y
      sudo systemctl enable apache2
      sudo systemctl start apache2
      echo "<h1>Welcome to my server </h1>" > /var/www/html/index.html
    EOF

    tags = {
        Name = "web-server"
    }

}

resource "aws_instance" "web-server" - This line declares a resource of type aws_instance and the name of this particular resource is "web-server".
ami = "ami-04b70fa74e45c3917" - This line specifies the Amazon Machine Image (AMI) to use for the instance. The AMI determines the operating system and other software for the instance.
instance_type = "t2.micro" - This line specifies the type of instance to launch. In this case, it’s "t2.micro", which is a general-purpose instance type.
availability_zone = "us-east-1a" - This line specifies the availability zone in which to launch the instance.
key_name = "main-key2"- This line specifies the name of the key pair to use for the instance. This key pair is used for SSH access to the instance.
The network_interface block within the resource associates the instance with the network interface that has been created earlier in the script with the name "web-server-nic".
The user_data block within the resource specifies a startup script to run on the instance when it launches. This script updates the package lists for upgrades and new package installations, installs Apache2, enables and starts the Apache2 service, and creates a simple HTML file as the index page for the web server.

Deploying the Infrastructure

Deploying the infrastructure to AWS is pretty straightforward. To do this:

Open a terminal in Visual Studio Code.
Ensure that it points to the directory containing the Terraform file.
Execute the following command: terraform apply If everything is set up correctly, the system will prompt you to enter “yes”. If there’s an error, it will point you to the location of the issue.

Next, open the AWS console and locate the EC2 instance that we just deployed.

Copy the IPv4 public address and paste it into a new browser tab. The HTML file that we deployed on our web server will be displayed.

Conclusion

So far, we’ve discussed how to deploy various resources on AWS using Terraform. We’ve also examined how these resources can be made dependent on each other, enabling them to form a unified infrastructure that serves a specific purpose in our case display a simple HTML page.

In an ideal situation, more than one instance would be deployed across different availability zones. Additional resources such as load balancers, S3 buckets, Content Delivery Networks (CDNs) like CloudFront, and Domain Name Services (DNS) like Route53, among others, would also be utilized.

All these components come together to create an infrastructure that is reliable, dependable, and operates smoothly.

I hope you won’t stop here. Continue exploring Terraform and even extend your knowledge beyond AWS to other cloud service providers. Good luck!

Provisioning Ansible on Ubuntu with Vagrant and Virtual box

Audu Ephraim — Sat, 27 Apr 2024 07:51:52 +0000

Introduction

Automation tools play a crucial role in day-to-day software development jobs. And it is on that note that Ansible has proven to be one of the most used technologies in today's software development world.

Ansible is an open-source software development tool that enables infrastructure as code (IaC). Ansible helps in:

Software provisioning - Ansible can prepare your servers with the necessary software and settings.
Configuration management - Ansible can manage system configurations to ensure your systems and servers are in the desired state
Application deployment - Ansible can automate the process of deploying applications to servers.

Ansible is agentless. It relies on temporary remote connections via SSH. Ansible uses a human-readable script called Playbook to automate tasks and manage and maintain system configurations.

Vagrant on the other hand is an open-source software that provides a convenient way to manage virtual environments making it easier to replicate virtual environments across different hosts.

Meanwhile, VirtualBox is a free open-source virtualization platform. Together with Vagrant, they provide a seamless solution for creating and maintaining virtual machines.
Objective

This article aims to guide us through the process of setting up a local environment with Ansible on Ubuntu with Vagrant and VirtualBox where we will run a simple Ansible playbook on the local machine to print “Hello World”.

Prerequisites

Software requirements

Ubuntu
Vagrant
Virtual Box Basic Knowledge of Linux commands

Setting up Ubuntu virtual machine with Vagrant and VirtualBox

To begin we will need to download and install Vagrant and VirtualBox

Installing VirtualBox Virtual Box is a free and open-source technology available for download for different operating systems.
Download VirtualBox: visit the VirtualBox download page to download the appropriate version for your operating system
Install virtual box: Install the virtual box application from the downloaded binary.
Installing Vagrant
Download Vagrant: Download Vagrant from the Vagrant download page
Install Vagrant: Follow the installation guide to install Vagrant for your specific operating system. Vagrant will be automatically added to your system path, so you can start using vagrant commands from your terminal.
Verify Installation: to verify that Vagrant was installed correctly execute the command below to see if it prints out the installed version.
vagrant -version

Creating an Ubuntu Virtual Machine

For the sake of this article, we will be using PowerShell. Run it as an administrator.

We will need to create a folder to store all our vagrant-related files, we will call the file. To do that run the following command.
mkdir ansibleProject
Cd into the directory with
cd ansibleProject
Initialise a vagrant file with an Ubuntu image. This will contain all the necessary configurations needed for our virtual machine. We are going to use the Ubuntu image available in the Vagrant cloud.
vagrant init
Start the virtual machine with the following command
Vagrant up

The above command will download the ubuntu/trusty64 virtual machine image from the vagrant cloud and start the VM.

It also creates a key pair so we can SSH into the machine once it is done setting up.

You can check the status of the virtual machine with the following command vagrant status. It should print out “running (virtualbox)”. we can also check the status by opening VirtualBox to see if our virtual machine is up and running.
SSH into the virtual machine by running the following command vagrant ssh

Installing Ansible

To install Ansible on our virtual machine running on Ubuntu we need to run the following commands.

sudo apt update
sudo apt install software-properties-common
sudo add-apt-repository --yes --update ppa:ansible/ansible
sudo apt install ansible

As soon as Ansible is done installing, we will verify if it was installed correctly.

To verify that Anisble was installed correctly run the following command
ansible –version.

Creating a playbook

To create a playbook, we will first create a new directory. This will contain our ansible playbook.
mkdir ansible-project
Create a new file called playbook.yml. This file will contain the task you want to perform on your machine.
Touch playbook.yml
Run ls to see if it was created successfully.
Write your playbook in the playbook.yml file

To do this:

Open the playbook.yml by running vi playbook.yml
Press “I” on your keyboard to enter Insert mode. Copy and paste the following:

---
- name: Basic playbook
  hosts: localhost
  tasks:
    - name: Print a message
      debug:
        msg: "Hello, World!"

Hit the “esc” key when done, to exit insert mode.
Type “:wq!” to exit and save the playbook.yml

This is a simple playbook script that tells Ansible to connect to a machine, in this case, localhost which is the machine Ansible is installed on.

Then tells the machine to print “Hello world”

run the playbook by running ansible-playbook playbook.yml

Conclusion

In this simple guide, we’ve seen how to set up a development environment on Ubuntu using Vagrant and VirtualBox, followed by installing Ansible for automation and configuration management.

With Ansible installed we have a powerful tool that can help reduce repetitive tasks and increase productivity.

We can take this further by automating configurations on the cloud with Ansible rather than on our local machines.

As we continue our journey, whether we are managing a small infrastructure or orchestrating a complex deployment Ansible simplicity makes it a great addition in our DevOps toolkit.
Don’t stop learning!

Harnessing App Flavors and Shared Preferences in Android: A Ride-Sharing App Case Study

Audu Ephraim — Sat, 02 Mar 2024 14:23:54 +0000

Introduction

In this article, we will explore the concept of app flavors in Android applications and how to customize these flavors. We will build a demo ride-sharing application with two flavors, one for the driver and one for passengers.

We will also be looking at saving a user's choice at the first launch with shared preferences so that, on the next launch, the user's choice will be remembered.

By the end of this article, you will have a solid foundational understanding of how to use app flavors in Android development and how to apply this knowledge to create more flexible applications.

Prerequisites

Basic knowledge of Android development, Gradle and Jetpack Compose is required to follow along with this article.

Understanding App Flavors

App flavors or build configurations are distinct ways to create distinct environments or “flavors” for your app, each representing a distinct variant of your application but from the same codebase.

App flavors are a way to implement different versions of an application with minor changes.

Common use cases of app flavors include:

Different versions of an application- you might have a free and premium version of your app. The free version could have ads and limited features, while the premium version has no ads and more features.
Geographical variations: if your app provides content or features based on geographical location, you can use app flavors.
Different UI for different user roles: in a ride-sharing app, like in our demo. You have one UI for drivers and one for passengers.
A/B Testing: you can use app flavors to create slightly different versions of your app for A/B testing. This can help you understand which features or designs users prefer.

Building different flavors for your app can be very useful, they also come with some level of complexity, so you need to think carefully about your needs before setting up a project to use app flavors.

If your separate apps require minor differences and theme changes but are still the same app, multiple flavors should be considered. However, if both apps require a lot of custom code differences you should rethink using app flavors.

Build Types

In the process of building Android apps, besides “flavors”, there’s another crucial concept known as “build types”.

A build type is responsible for defining settings related to the app’s build and packaging process, such as whether it’s debuggable and what signing keys to use. The standard build types are “debug” and “release”.

On the other hand, a flavor is used to specify features, devices and API prerequisites like custom code and layout, as well as minimum and target API levels, among other things.

The term “Build Variant” is used to describe the combination of build types and flavors.

For every combination of a defined flavor and build type, there exists a corresponding build variant. In the case of our ride-sharing application, we will have these corresponding build variants.

driverAppDebug
driverAppRelease
passengerAppDebug
passengerAppRelease

Therefore, it is important not to mix up the concept of app flavors, build types and build variants with each other as each has its unique role to play.

Let’s get started!!!

Creating the project

We do not need to do anything special to create our project. We’ll create a new regular Android project and call it myRideSharingApp and wait for the project to be done building.

Setting up Project Flavors

There are two ways to do this.
One way is to click on the build option in Android Studio and select “edit flavors”

We can now create a flavor dimension, this is essential when creating multiple flavors as each flavor belongs to a dimension. We can also edit applicationID, add a signing config among other things.

Then click “Okay” A Gradle sync will occur and when it is done our flavors will be created.

Another method to create our flavors involves opening our module’s build.gradle file. Inside the Android block of this file, we can add the specifications for our flavors.

flavorDimensions += listOf("driver", "passenger")
productFlavors {
   create("driverApp") {
       dimension = "driver"
       applicationId = "com.example.myridesharingapp.driver"
       applicationIdSuffix = ".driver"
       versionCode = 1
       versionName = "1.0"
       versionNameSuffix = "-driver"
       targetSdk = 34
       minSdk = 24
   }

   create("passengerApp") {
       dimension = "passenger"
       applicationId = "com.example.myridesharingapp.passenger"
       applicationIdSuffix = ".passenger"
       versionCode = 1
       versionName = "1.0"
       versionNameSuffix = "-passenger"
       targetSdk = 34
       minSdk = 24
   }
}

Let's try to understand what's going on:

flavourDimension+= listOf("driver", "passenger"): This line creates dimensions of your product flavors. Each dimension represents a different characteristic that a product flavor can modify. In this case we have two dimensions: driver and passenger.
productFlavors: In this block, you define your product flavors. Each product flavor is a variant of your app that you can build
create("driverApp") { ... } and create("passengerApp") { ... } these blocks define the driverApp and passengerApp flavors respectively. Each block sets properties specific to that flavor
dimension: This property sets which flavor dimension this product flavor belongs to
applicationId: This property sets the unique ID for this version of the app. It’s used by the Android system to identify your app among all others. Two applications with the same applicationID cannot exist on the same device.
applicationIdSuffix: This property adds a suffix to the applicationId for this flavor. This allows you to install multiple flavors of the app on the same device.
versionCode: This property sets the version code for this flavor. The version code is an integer value that represents the version of the app.
versionName: This property sets the version name for this flavor. The version name is a string that represents the version of the app.
versionNameSuffix: This property adds a suffix to the versionName for this flavor.
targetSdk: This property sets the target SDK version for this flavor. The target SDK version is the latest Android version that your app has been tested with.
minSdk: This property sets the minimum SDK version for this flavor. The minimum SDK version is the lowest Android version that your app supports
Once done, sync the project and we've created our flavors.
Note: All these details and specifications can be entered using the ‘Edit Flavor’ dialog.

File Structure

Every flavor file structure should look exactly like the main directory file structure. In our case it should look like this:

Every flavor should have its res folder that should contain values specific to each flavors like drawables and the like.

Also, each flavor can have its manifest.xml file to specify settings and configurations particular to each flavor.

For example, you might have different services or permissions for the driverApp flavor compared to the passengerApp flavor.

These differences would be reflected in each flavor’s respective AndroidManifest.xml file.
It’s a powerful feature that adds a lot of flexibility to your app’s configuration.
Each flavor should contain its own MainActivity.kt file. In our case passengerActivity.kt and DriverActivity.kt serve as an entry point to each application (Flavor).

Setting Up Activities and Manifest Files

We create our driverActivity.kt in our driverApp directory which contains a simple composable screen that indicates that this is the driver screen.

class DriverActivity : ComponentActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContent {
            DriverScreen()
        }
    }
}

@Composable
fun DriverScreen() {
    Box(contentAlignment = Alignment.Center) {
        Text(
            text = "This is the driver Screen",
            fontSize = 20.sp
        )
    }
}

In our passengerApp directory, we do the same: we create a passengerActivity and a composable to indicate that this is the passenger screen.

class PassengerActivity : ComponentActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContent {
            PassengerScreen()
        }
    }
}

@Composable
fun PassengerScreen() {
    Box(contentAlignment = Alignment.Center) {
        Text(
            text = "This is the passenger Screen",
            fontSize = 20.sp
        )
    }
}

After we are done setting up our passengerActivity and driverActivity, we need to declare each activity in its manifest file so that the Android systems know about it. The passenger manifest file will look like this:

<manifest xmlns:android="http://schemas.android.com/apk/res/android">

    <application>
        <activity android:name=".PassengerActivity"
            android:exported="true">
        </activity>
    </application>

</manifest>

and the driver's manifest file will look like this:

<manifest xmlns:android="http://schemas.android.com/apk/res/android">

    <application>
        <activity android:name=".DriverActivity"
            android:exported="true">
        </activity>
    </application>
</manifest>

The system must know that this activity is part of your app so it can start the activity when requested to do so through an intent.

Navigating to Individual Flavors

In our mainActivity.kt file in our main directory we are going to create a simple composable with two buttons to navigate to our desired application (Flavor).

@Composable
fun SharedComponents(
) {
   val context = LocalContext.current
   Box(
       contentAlignment = Alignment.Center,
       modifier = Modifier.fillMaxSize()
   ){
       Column(
           modifier = Modifier.fillMaxSize(),
           horizontalAlignment = Alignment.CenterHorizontally,
           verticalArrangement = Arrangement.Center
       ) {
           Text(
               text = "This is the shared module",
               fontSize = 20.sp
           )
           Spacer(modifier = Modifier.height(30.dp))
           Button(onClick = {
               val intent = Intent(context, DriverActivity::class.java
               )
               context.startActivity(intent)}
           ) {
               Text(text = "driver")
           }
           Spacer(modifier = Modifier.height(30.dp))
           Button(onClick = {
               val intent = Intent(context, PassengerActivity::class.java)
               context.startActivity(intent)
           }) {
               Text(text = "passenger")
           }
       }
   }
}

Upon running the application, you should be able to navigate to any of the flavors by clicking a button. If you close the application and reopen it, you’ll notice that it will return to the screen of the shared module.

This current setup is not ideal. We would prefer our application to remember our selection. That way, the next time we open our application, we will be directed to the flavor of our initial choice.

To achieve this, we will be using shared preferences.

Setting up Shared Preferences

We are going to create an object called preference in our main module and add the following.

object Preference {

   private const val PREFERENCE_NAME = "app_preference"

   private const val USER_CHOICE = "user_choice"

   fun saveUserChoice(context: Context, choice: String){
       val sharedPref = context.getSharedPreferences(PREFERENCE_NAME, Context.MODE_PRIVATE)
       sharedPref.edit().putString(USER_CHOICE, choice).apply()
   }

   fun getChoice(context: Context): String?{
       val sharedPref = context.getSharedPreferences(PREFERENCE_NAME, Context.MODE_PRIVATE)
       return sharedPref.getString(USER_CHOICE, null)
   }
}

We create two functions to save and get our user’s choice.
In our mainActivity class we retrieve a user’s choice from sharedPreference and launch the appropriate activity when the app starts.

class MainActivity : ComponentActivity() {
   override fun onCreate(savedInstanceState: Bundle?) {
       super.onCreate(savedInstanceState)
       setContent {
           val userChoice = Preference.getChoice(this)
           MyRideSharingAppTheme {
               Surface(
                   modifier = Modifier.fillMaxSize(),
                   color = MaterialTheme.colorScheme.background
               ) {
                   if (userChoice == "driver"){
                       val intent = Intent(this, DriverActivity::class.java)
                       startActivity(intent)
                       finish()
                   }else if(userChoice == "passenger"){
                       val intent = Intent(this, PassengerActivity::class.java)
                       startActivity(intent)
                       finish()
                   }else{
                       setContent {
                           MyRideSharingAppTheme {
                               Surface(
                                   modifier = Modifier.fillMaxSize(),
                                   color = MaterialTheme.colorScheme.background
                               ) {
                                   SharedComponents()
                               }

                           }

                       }

                   }
               }
           }
       }
   }
}

We also modify our onClick lambda to be able to save a user's choice at the click of the button.

Button(onClick = {
   **Preference.saveUserChoice(context, "driver")**
   val intent = Intent(context, DriverActivity::class.java
   )
   context.startActivity(intent)}
) {
   Text(text = "driver")
}
Spacer(modifier = Modifier.height(30.dp))
Button(onClick = {
   **Preference.saveUserChoice(context, "passenger")**
   val intent = Intent(context, PassengerActivity::class.java)
   context.startActivity(intent)
}) {
   Text(text = "passenger")
}

Run the application and make a selection. After closing and reopening the application, you’ll find that it remembers your initial choice, just as intended!

Conclusion

So far, we’ve discussed app flavors and their use cases. We’ve examined how to set up app flavors and customize each one to meet specific requirements.

We’ve also explored how to establish file structures and navigate to a particular app flavor.

Finally, we’ve looked at how to save a user’s preferred flavor with shared preferences for subsequent runs.

In conclusion, the concept of app flavors in Android development offers a powerful and flexible way to manage different versions of an application.

Through the example of a ride-sharing app, we’ve seen how to create and customize ‘driver’ and ‘passenger’ flavors, each tailored to meet specific user needs.

We’ve also explored how to use shared preferences to enhance the user experience by remembering their preferred flavor.

This not only streamlines the development process but also provides a more personalized user experience. Happy Coding!

Questions and suggestions would be highly appreciated.

You can find the link to the whole project Here

Building Blocks: A Beginners Guide to REST APIs with Kotlin and Ktor

Audu Ephraim — Fri, 26 Jan 2024 11:16:26 +0000

In the world of software development, Rest APIs have emerged as fundamental building blocks of modern web applications.

If you're a novice or an experienced developer, looking to experience new territories, this guide is your starting point for understanding and creating REST APIs using Kotlin and Ktor.

Kotlin is a modern language that offers more concise syntax than Java. ktor is a Kotlin framework for building asynchronous servers and clients are gaining a lot of traction in the tech industry.

This powerful duo simplifies building lightweight, scalable, and efficient REST APIs.

In this guide, we’ll create a basic REST API. This API will return a list of countries and their details. It will also allow us to view information for a specific country and add a new country with its details.

Our goal is to simplify the concept of REST APIs and provide a clear, step-by-step method to build one using Kotlin and Ktor. So, let’s get started!

Prerequisites

Basic knowledge of Kotlin and basic HTTP Knowledge

What is a REST API: A Brief Introduction

An API(Application Programming Interface) is a set of rules and protocols that define how applications, devices, and services can communicate with themselves over the Internet using different types of requests to get different types of information.

A REST(REpresentation Stateful Transfer) API is an API that conforms to the design principles of the REST architectural style. You can read more about this architecture and its principles HERE.

KTOR

As we earlier said, Ktor is a Kotlin framework for building asynchronous server-side and client-side applications with ease. It is entirely written in Kotlin and supports coroutines for asynchronous programming.

Setting Up Ktor and Kotlin

To start working with Ktor and Kotlin we need to set up our development environment.

To do that we will be making use of IntelliJ IDEA which is the leading Kotlin and Java IDE you can get it Here. You can download the ultimate edition which is paid or you can just get the community edition which I will be using. After downloading just follow the prompts to install.

To get started with Ktor, we need to head over to the Ktor project generator, if you are using the Ultimate Edition of IntelliJ you will be able to create your Ktor project right from inside your IDE.
In the Ktor project generator, you should see a page like this:

Give a project name of your choice, In this article, we will name our project "SampleKtorProject"
Click on Adjust Project Settings, and you will see a drop-down with a bunch of options that should help you customize your project even more. But for the sake of this guide, we will leave everything as it is.
Then click on Add plugins These plugins allow us to have access to specific back-end behaviors that we typically use in backend frameworks. If you scroll down the list, you should see all available plugins, but for this project, you will be adding just four plugins.

Routing- allows us to define the routes we respond with data. In our case countries
ContentNegotiation- used for JSON parsing. When we respond with a Kotlin data class it will automatically be parsed to a JSON
CallLogging- this will just log when a client requests a server so we can see what's happening.
Kotlinx.serialization- which is used by the content negotiation for serializing and deserializing the data
When we are done you can go ahead and click on generate project, which automatically downloads the project in a zip file. Unzip and save in any location of your choice, then open the project in IntelliJ. Wait for the project to finish building.

Let’s go over to the project

Project Structure

We have the build.gradle.kts file. It is the Kotlin version of Gradle. Gradle is a build automation tool, if you are coming from an Android background this should be familiar. It contains our project dependencies and some project settings.

Navigate to src->main-> Kotlin in which we have our Application.kt file which is the main entry point of our program. Similar to mainActivity in Android. In this file, we have our main function as we know from plain kotlin.

head over to the plugins folder, here you will see all the plugins we added in our initial project setup, which created some default files
In the call Monitoring.kt, you’ll see a lambda block where you can customise your logging. Like how detailed you want your logs to be and if you want to filter your log messages.
In the Routing.kt file this is where the routes are defined, that our server responds at. Here we define and configure our routes.

To test that everything is working perfectly, head over to Application.kt, and click on the run button beside fun main(){...} to start the project, head over to any browser of your choice and hit the address returned in your IntelliJ console.

This would return “hello world” as seen in your routing file. Now that we are familiar with our project, let’s start building.

Since this is a very simple project we won't go deep into structuring our project.

Let us create two new packages in our root folder, Let's call them “model” and “routes” To do this:

right-click on your root package->new->package
Inside the model package let's create a new data class and call it “country”. In our API each country has an id, a name, a continent, and a capital. The data class will end up looking like this:
```
@Serializable
data class Country(
val id: String,
val name: String,
val continent: String,
val capital: String
)
```

The “@Serializable” annotation enables us to parse this content of data class content to JSON and send it over the network.

Next inside the route package create a new file, named countriesRoute.kt, and add the following code.

private val countries = mutableListOf<Country>(
   Country("1","Japan", "Asia", "Tokyo"),
   Country("2","Egypt", "Africa", "Cairo"),
   Country("3","Canada", "North America", "Ottawa"),
   Country("4","Brazil", "South America", "Brasilia"),
   Country("5","New Zealand", "Oceania", "Wellington"),
   Country("6","England", "Europe", "London"),
)

fun Route.countries(){
   route("/countries"){
       get{
           call.respond(
               HttpStatusCode.OK,
               countries
           )
       }

       get("{id?}"){
           val id = call.parameters["id"]?: return@get call.respondText(
               "missing id",
               status = HttpStatusCode.BadRequest
           )

           val country = countries.find { it.id == id }?:return@get call.respondText(
               "country with id $id not found",
               status = HttpStatusCode.NotFound
           )
           call.respond(country)
       }

       post{
           val country = call.receive<Country>()
           countries.add(country)
           call.respondText("country Added ", status = HttpStatusCode.Created)
       }
   }
}

Let’s go over it:

The data that we return to a client upon a request does not originate from a database, we create a mutable list of Country objects. Each country object has an id, name, continent, and capital.
fun Route.countries(): It is an extension function of the route class in kotlin. It is a way to group related routes.
route("/countries"): It sets up a route group called countries with the path /countries. All routes defined within the block will have /countries as their base path.
GET: It sets up a get request. When a get request is made to /countries, the server responds with a status code of ok and a list of countries.
get("{id?}"): This sets up another get request, but this one expects an optional id parameter in the URL. If an ID is provided e.g. countries/1. It tries to find a country with that id. If it finds one it responds with the country data; otherwise, it responds with a not found status and the message “country with id not found”. If no ID is provided for example: /countries/ it responds with a bad request status code.
Post: this sets up a POST request. It expects the request to contain a country object. It adds the new country to the countries list and responds with a created status and a confirmation message

Registering routes

Now all we have to do is head on to our plugin package in the Routing.kt file, remove the default, hello world get request, and call countries() like this:

fun Application.configureRouting() {

   routing {

       countries()

   }

}

Testing with Postman

Our API is ready for testing. We are going to use Postman to see how it works.
Postman is a tool for testing API endpoints and seeing how it works. You can get it here.

Once you’ve downloaded, installed, and launched Postman, you should see a window like this:

click on the + to open a new tab.
After that, head back to IntelliJ, and launch your server. Once it is up and running without any errors, return to Postman.

Testing GET requests

First, let's test out GET to return the list of every country and details about them. To do this:

Select GET
In the empty text field, put the address we want to request, in our case, it is http://127.0.0.1:8080/countries then hit send.

You should see a response like this:

Great!! Our get request is working properly.

Let's try and get a country with ID 3. Edit the URL, to look like this http://127.0.0.1:8080/countries/3 and hit send.

You should get a response like this:

Testing Post request

Change the request type to POST
Edit the URL to http://127.0.0.1:8080/countries
Select “body” and “raw” in the options below the URL text field
Enter the data of the country you want to add

Click send, and you will see a response “country Added”

Change the request type to get and send the request, you should see the newly added country

Our API is now working and it's returning the desired results.

Conclusion

In this article, we've taken our first steps into the world of API development with Kotlin and Ktor. We've seen how Kotlin's concise syntax and Ktor's robust features can make the process of building APIs more efficient and enjoyable.

We started with the basics of setting up a Ktor project and gradually moved on to creating our first API endpoints with Ktor. We've also learned about key concepts such as routing and request handling.

However, this is just the beginning. There's much more to explore in the realm of Kotlin and Ktor, such as authentication, database integration, and testing. As you continue your journey, remember that the key to mastering any technology is consistent practice and curiosity.

So, keep building, keep exploring, and most importantly, have fun doing it.

Happy coding!