DEV Community: Ivan Camacho

Self-host ASP.NET DevOps build server (K3S, Docker, GitHub Actions, ArgoCD)

Ivan Camacho — Mon, 30 Oct 2023 17:11:54 +0000

Objectives

My objectives when setting up K3s in my home lab were to play around with Kubernetes and save on cloud costs by utilizing the computing resources to host CI runners for GitLab and GitHub. However, once complete, I decided it'd be fun and good practice to configure a deployment system linked to my K3S cluster as well.

This article is to possibly help anyone also interested in learning Kubernetes and setting up a CI/CD pipeline while saving on costs.

Here are the steps I took!

Preparation

Proxmox & K3s -

I chose Proxmox as a hypervisor because it simplifies resource management and enables stable, scalable local Kubernetes environments.
I'm using K3s because it offers Kubernetes features like scalability and security while using fewer resources and being quicker to deploy than K8s.

MetalLB -

MetalLB is a solution to ensure services within a locally hosted Kubernetes cluster are accessible, load-balanced, and have their networking traffic adequately managed.
Although MetalLB isn't recommended for production environments, it works well for this project as the objectives are focused on testing and development.
For information on setting up MetalLB, take a look at their GitHub page here - https://github.com/metallb/metallb

TechnoTim has a great article & video covering how to automate the set-up and deployment of both K3s and MetalLB using Ansible: https://technotim.live/posts/k3s-etcd-ansible/

Steps

Example Code :
App CI Repo ~ https://github.com/swe-pea/ASP-NET-TestApp
Kubernetes Manifest/CD ~ https://github.com/swe-pea/home-lab/tree/main/ASP-TestApp

Setting Up The ASP.NET CI

I started off by creating a GitHub repo dedicated to storing application code and building new containerized versions of the application.
To self-host a single GitHub runner for the CI repository, I first set up an ACR (Actions Runner Controller) on K3s by applying the following manifest and registering a GitHub PAT as a Kubernetes secret

# Applies kubernetes config to set up ARC
kubectl apply -f https://github.com/summerwind/actions-runner-controller/releases/latest/download/actions-runner-controller.yaml

kubectl create secret generic controller-manager \
    -n actions-runner-system \
    --from-literal=github_token=${GITHUB_TOKEN}

Then, create and apply the following .yml file. By creating the secret and applying the following manifest, a single GitHub runner is created and linked to the CI repo.

# runner.yaml
apiVersion: actions.summerwind.dev/v1alpha1
kind: Runner
metadata:
  name: example-runner
spec:
  repository: exampleUser/example-repo
  env: []

Additionally, I created a DockerHub repository to store the application docker images built from the GitHub repository.

The GitHub workflows pushed with the application code automatically run on the newly configured self-hosted runner. Using docker/login-action@v3 and docker/build-push-action@v5, the containerized application is stored in DockerHub.

Deploying The ASP.NET Application

Having successfully set up the CI pipeline and containerized the ASP.NET application, it's time to deploy it to the K3S cluster.

Deployment Repository: I created a second GitHub repository with the sole purpose of deploying the ASP.NET application. This repository will use Kubernetes manifests (in .yml format) to define how the application should operate within the K3S cluster(replica count, ports, etc.)
ArgoCD: Using a deployment tool like ArgoCD makes it seamless to connect repositories to your Kubernetes cluster and deploy applications. Although I won't go into the details of how to set ArgoCD up, as a Kubernetes amateur, this is the method I went with for quickly deploying my Kubernetes manifest from the repository to my cluster.

Helm Charts: Although Helm is a popular package manager tool for Kubernetes, with Helm charts offering pre-configured packages to make deployment quick and consistent, I decided to work with Kubernetes manifests directly to focus on the fundamentals.
Continuous Deployment: Since the project objectives weren't focused on deployment strategies like rolling update, canary, or blue-green, continuous deployment was not configured. However, you can create a continuous deployment from this solution by using webhooks if you are interested.

Conclusion

In conclusion, this journey has provided insights into setting up a cost-effective, self-hosted CI/CD pipeline in a home lab, utilizing K3S and other OSS. I've learned a lot while also having lots of fun, so I'm hoping by sharing my experience working on this small project, someone else can maybe gain some inspiration or value from it as well!

Happy deploying,

-Ivan

Cloud Resume Challenge - Azure Edition

Ivan Camacho — Sun, 10 Sep 2023 01:19:15 +0000

While looking through articles from the dev community, I recently stumbled across a member's blog post describing their experience completing "The Cloud Resume Challenge." This caught my eye as the challenge looked fun and was similar to a simple personal blog project I had already wanted to start. After reviewing The Cloud Resume Challenge website, I knew this project was perfect for me!

Link To Challenge: The Cloud Resume Challenge
Link To End Result: https://www.cuauh.codes/
GitHub: https://github.com/swe-pea/AzureResumeChallenge

Considering Tools, Goals, & Objectives

Before building things, I took a moment to assess the project requirements and weigh different technology solutions while considering my objectives. Off the bat, I knew I would be using Azure as the cloud provider (hence the title) since I've been heavily Azure-focused with most recent projects, and my only other cloud experience has been with AWS (Eliminating GCP - as learning a new provider doesn't align with my objectives for the project).

Consideration for version control & CI/CD fell between Gitlab, GitHub, and Azure DevOps. With web development taking priority for "area of desired learning/growth" with this project, I went with what I'm currently most comfortable & experienced working with: Azure DevOps. However, with the ease of authentication/compatibility, I used GitHub for the repo, with the "Azure Pipelines" GitHub App integration. I recommend this solution when working with Azure. However, I'll probably be using more GitHub actions moving forward.

Finally, rather than using Python to create the "Visitor Counter API" as recommended, I enjoy coding C# a lot and therefore opted to use that, although it's arguably overkill for this project, and I didn't do any fancy OOP stuff.

Setting Up The Site

Some of the most time-consuming pieces of this project for me were in the beginning with simply building & setting up the static website and getting the visitor counter to work correctly. It has been a long time since I've touched HTML, CSS, and Javascript. My goal was to keep everything as simple as possible & keep things secure, which looked slightly different as I altered the challenge to host my blog site rather than a single-page resume.

I used GoDaddy to purchase my custom domain name & another site for the SSL certificate, where, in retrospect, I could've used Azure's internal services to buy a custom domain and configure it with a managed SSL certificate to minimize complexity. It wasn't a huge issue; I made a CNAME record mapping from the DNS provider I used to my Azure CND endpoint. Once that propagated, my Azure Front Door & CDN profile was ready to rock & roll!

Site Visitor Counter (DynamoDB & Azure Functions)

After writing the javascript on my site for how I wanted the "Visitor Count" feature to work, I had a good idea of implementing the API using Azure Functions to communicate securely with a simple DB table entity. Visual Studio IDE helps create an Azure Function app, a speedy and seamless process, so you can focus on writing code rather than overspending time on configuration/authentication. I highly recommend it if working with Azure!

Using VS to create C# Azure Function AppUsing the Cosmos DB .NET SDK, and after a period of trial & error, I had my visitorcounter.js method using the HTTP trigger to increment and display my site counter. What was most challenging for me was getting my visitorcounter.js logic to work with my site the way I wanted. Luckily, with the abundance of online resources developers have today (forums, AI tools, etc.), overcoming obstacles for integrating the front end with the back wasn't nearly as challenging as it would've been "back in the day." I have tons of gratitude for the easily accessible tech resources available in today's day & age!

Infrastructure As Code & Automation (CI/CD)

Front End Deployment Stg in Azure DevOpsIt could defeat the purpose of IaC for a project like this; however, I've always automated processes/tasks only after understanding the task "inside and out," and so I provisioned & configured my resources using Azure CLI and the UI portal first to get a POC(Proof of concept) working. I always learn a lot from building out and testing a little POC; in this case, it also helped me consider (in terms of automation) how robust, flexible, and well-designed a solution I wanted to create.

For IaC, I was solely going to use Terraform. However, creating an Azure Document DB Table API (Supports enabling a "free-tier" feature) was difficult as there's no "Document DB" module in Terraform. So, I resorted to using Bicep to handle that. I had gotten confused with all the different APIs Cosmos DB supports and wanted to select the most cost-effective one.

TDLR ~ Review

My primary goals for this project were to gain experience with web development, write this blog, satisfy the challenge requirements, and get an MVP launched. Sticking more or less to these goals left tons of room for improvement, especially with the backend of the project, specifically the visitor counter feature (DB & Az Function) and the automation pipelines.

That said, however, I thoroughly enjoyed working on this challenge (even if altered a little bit) and highly recommend it to just about anyone interested in technology, as it touches many different areas! I have already recommended it to several people. It's been a perfect and incredibly fun "breaking the ice" project to get kicked off developing experience & comfort writing "dev"/technical blog posts and sharing projects online despite their flaws!

I am excited to continue writing blog posts on tech-related stuff geared toward helping others and providing value. While also sharing more public projects on Github and connecting with other like-minded techies! I am very grateful for the developer communities; thanks.
-Ivan