DEV Community

Siddharth Bhamare
Siddharth Bhamare

Posted on

End-to-End CI/CD Setup for .NET Microservices on AWS EKS Using Azure DevOps

#kubernetes #devops #aws #azure

Create the ecosystem connections between Azure DevOps, AWS, and your repo first, and understand how every piece (Dockerfile, YAML, Kubernetes manifests, etc.) in terms of cloud infrastructure. Understand CI/CD, Cloud compute service EKS, and API Gateway for your microservice (NGINX)

Overall Cloud Architecture (Visual Overview)

🧭 PART 1 — The Big Picture Flow (Visual Overview)

Here’s the end-to-end relationship of files, pipelines, and services:

                ┌──────────────────────────────────────────────────┐
                │                 Azure DevOps                     │
                │──────────────────────────────────────────────────│
                │                                                  │
                │ 1️⃣ Build Pipeline (azure-pipelines.yml)          │
Developer Push  │    ↓                                             │
     Code  ────▶│  (uses Dockerfile + dotnet test + code coverage) │
                │    ↓                                             │
                │  Builds Docker image & pushes to AWS ECR         │
                │    ↓                                             │
                │  Publishes artifacts (manifests/templates)       │
                │                                                  │
                │ 2️⃣ Release Pipeline / Deploy Stage               │
                │    ↓                                             │
                │  (takes templates → replaces variables → applies)│
                │    ↓                                             │
                │  kubectl apply to AWS EKS (Kubernetes cluster)   │
                └──────────────────────────────────────────────────┘
                                     │
                                     │  Deploys Container Image
                                     ▼
                 ┌───────────────────────────────────────────┐
                 │                AWS EKS Cluster             │
                 │───────────────────────────────────────────│
                 │ Deployment.yaml → creates Pods             │
                 │ Service.yaml → creates ClusterIP/LoadBalancer │
                 │ ConfigMap/Secrets → inject env vars         │
                 │ Ingress.yaml (NGINX) → exposes API          │
                 └───────────────────────────────────────────┘
                                     │
                                     ▼
                        🌐 Accessible API via Ingress
                          Logs → Application Insights
Enter fullscreen mode Exit fullscreen mode

🧩 PART 2 — How Files Connect (Step-by-Step Flow)

Let’s link each file and configuration together in order:

Step File / Component Purpose Consumes / Produces
1 Dockerfile Defines how to build your .NET app into a container. Consumed by build pipeline (docker build)
2 azure-pipelines.yml Your CI/CD automation definition. It:
- Builds app
- Runs tests + coverage
- Builds Docker image
- Pushes image to AWS ECR
- Deploys manifests to AWS EKS		 Consumes: Dockerfile, test project, manifests.
Produces: Docker image in ECR, deployment in EKS
3 configmap.template.yaml / secrets.template.yaml Define environment variables (non-sensitive in ConfigMap, sensitive in Secret). Consumed by Deploy stage. Gets substituted and applied to Kubernetes.
4 deployment.yaml.template Defines how many Pods to run, which image to use, and what ConfigMap/Secret to pull. Consumed by Deploy stage. Gets image tag replaced and applied to EKS.
5 service.yaml Exposes your app internally (ClusterIP / LoadBalancer). Linked to Deployment via label selectors.
6 ingress.yaml Exposes app to the public through NGINX Ingress Controller. Linked to Service. Controls routing and domain mapping.
7 App Insights config (via environment variable) Captures logs from the app and sends to Azure. Linked via Secret/ConfigMap and Serilog in code.

⚙️ PART 3 — What to Set Up First (Foundation Setup Order)

Here’s the correct first-time setup order for Azure DevOps ↔ AWS ↔ Repo.

🧩 Step 1: Setup AWS Resources

Resource Action Why
ECR (Elastic Container Registry) Create a repo (e.g., myservice). Stores your Docker images built by Azure DevOps.
EKS (Elastic Kubernetes Service) Create a cluster (via eksctl or AWS Console). This will host your deployed containers.
IAM User/Role for Azure DevOps Create an IAM user with permissions for ECR (push/pull) and EKS (read/write). Generate Access Key + Secret Key. Used by Azure DevOps to authenticate with AWS.

✅ Tip: Attach AWS managed policies:

  • AmazonECRFullAccess
  • AmazonEKSClusterPolicy
  • AmazonEKSWorkerNodePolicy
  • AmazonEKS_CNI_Policy
  • AmazonEC2ContainerRegistryPowerUser

🧩 Step 2: Setup Azure DevOps Project

  1. Go to dev.azure.com → create Project (e.g., MyServiceProject).
  2. Create a Service Connection for AWS:
  • Project Settings → Service connections → New → AWS.
  • Enter Access Key, Secret Key from Step 1, Region.

  • Name it e.g., AWS-Prod-Connection.

    1. Create Pipeline:

  • Go to Pipelines → New → Import azure-pipelines.yml from your repo root.

    1. Create Environment Variables / Pipeline Variables:

  • APP_ENV, DB_CONN, FEATURE_FLAG_X, etc.

    1. Create Variable Groups (optional) for environment-specific settings.

🧩 Step 3: Setup Repo and Link to Azure DevOps

  1. Push your repo (code + pipeline + k8s templates) to Azure DevOps Repos or GitHub.
  2. Ensure your pipeline YAML file is at repo root.
  3. In Azure DevOps → Pipeline → Create pipeline → Choose “Existing Azure Pipelines YAML file”.

🧩 Step 4: Setup Kubernetes Connectivity

Inside Azure DevOps pipeline, we run commands like:

aws eks update-kubeconfig --region <region> --name <EKS_CLUSTER_NAME>
Enter fullscreen mode Exit fullscreen mode

🔑 This works only if your AWS Service Connection user has EKS cluster permissions.

You can test connectivity manually on your local machine:

aws eks update-kubeconfig --region <region> --name <EKS_CLUSTER_NAME>
kubectl get nodes
Enter fullscreen mode Exit fullscreen mode

If it works locally, it’ll work inside the pipeline.

🧩 Step 5: Setup NGINX Ingress Controller

Once the EKS cluster is ready:

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx --create-namespace
Enter fullscreen mode Exit fullscreen mode

It’ll deploy the NGINX ingress controller.
Then your ingress.yaml connects external traffic to your app’s service.yaml.

🧩 Step 6: Setup Application Insights (Azure Portal)

  1. In Azure Portal → Create a Log Analytics workspace + Application Insights.
  2. Copy its Connection String.
  3. Add that value as a Kubernetes Secret or pipeline variable (to inject during deployment): 
   APPINSIGHTS_CONNECTIONSTRING="InstrumentationKey=..."
Enter fullscreen mode Exit fullscreen mode
  1. Your .NET app automatically starts sending logs once deployed.

🧩 Step 7: Run the Full Flow

✅ Push code to main → Build pipeline triggers →
✅ Docker image built + pushed to ECR →
✅ Deploy stage runs →
✅ kubectl apply updates EKS →
✅ NGINX Ingress exposes service →
✅ Application logs flow to AppInsights.

🔁 Understanding Template Variables

Let’s clarify what to replace in your .template files:

Placeholder Replace With Comes From
__IMAGE__ <AWS_ACCOUNT_ID>.dkr.ecr.<region>.amazonaws.com/myservice:<Build.BuildId> Build pipeline
__APP_ENV__ Production / Staging Pipeline variable
__FEATURE_FLAG_X__ true/false Pipeline variable
__DB_CONN__ PostgreSQL connection string (from AWS RDS / Secrets Manager) Pipeline variable / Secret
__APPINSIGHTS_CONNECTIONSTRING__ Value from Azure Application Insights Secret
__EKS_CLUSTER_NAME__ Your EKS cluster name AWS Console

🧩 BONUS — Suggested First-Time Order of Execution

Step Action Where
1 Create AWS ECR repo AWS Console
2 Create AWS EKS cluster AWS Console / eksctl
3 Setup IAM User with ECR/EKS permissions AWS IAM
4 Setup Azure DevOps Project + AWS Service Connection Azure DevOps
5 Push repo (code + Dockerfile + pipeline YAML + k8s templates) Git
6 Create pipeline (YAML) and run once manually Azure DevOps
7 Verify ECR image pushed AWS Console
8 Verify EKS deployment created via kubectl get pods Local or DevOps log
9 Install NGINX ingress, create ingress.yaml AWS EKS
10 Add Application Insights connection string & verify logs Azure Portal

Top comments (0)