DEV Community: Timur Galeev

memex: training for the Forward Deployed Engineer era with a private memory brain on AWS

Timur Galeev — Fri, 10 Jul 2026 18:14:13 +0000

Or: what happens when an engineer gets tired of explaining himself to his own tools.

Before AI, and after

First, the way this job used to look.

A few years ago, if a company wanted a custom internal tool, the path was long: workshops, a requirements document, a team, a budget, a quarter. I sat somewhere in the middle of that as a cloud engineer - building the infrastructure, watching good ideas die in handover documents. Knowledge lived in people's heads and in wikis nobody read. When someone left, their know-how left with them.

Then AI tooling got serious, and the math changed. One person who understands both the business problem and the cloud can now sit down with a company, find the process that quietly eats two months a year, and ship a working tool for it in days. Not a slide deck - a running application.

There's a name floating around for this way of working: Forward Deployed Engineer. Someone who embeds with a company, learns how the place actually works - not how the org chart says it works - and deploys AI where it hurts. Let me be precise, because colleagues read this blog too: that is not my job title, and it's not a description of my day job. It's a hat I've been trying on for about half a year in my own time - side projects, weekend builds for friends' businesses, endless experiments - because I think it's where this profession is heading, and I'd rather learn the way of working before it becomes the job description.

The most fun engineering I've done. Also a memory shredder.

One month my evenings are full of warehouse logistics for a friend's company. The next it's a time-tracking tool. Then a Kubernetes upgrade study. Whatever belongs to an employer or to someone else's business stays there - that's the deal, and I don't bend it. But everything around the work is mine to keep: the architecture patterns that held up, the AWS traps I walked into, my side projects, my notes, the half-finished idea I mumbled into a voice memo at 11pm.

And here's the embarrassing part. I help companies stop losing knowledge, while my own knowledge lived in twelve places at once. An Obsidian vault. Git history. Old chat sessions. Voice memos. My head, allegedly. Every new AI session started from zero, and there I was again - re-explaining my own infrastructure to a tool that had helped me build that same infrastructure the day before.

At some point that stopped being funny. So I built memex.

What memex is, in plain words

memex is a private memory service. It runs in my own AWS account, it holds whatever I decide to feed it - notes, project docs, source code, and in my own install even my mail and calendar (that part is my personal wiring, not something the public repo ships) - and any AI tool I use can ask it questions.

If you don't write code, here's the version I'd tell over coffee: imagine a new colleague who has read every note you ever wrote, never forgets any of it, never leaves the company, and answers any question in two seconds - always quoting where they read it. That colleague costs less than your coffee budget. That's memex.

The technical version: the asking happens over MCP. MCP (Model Context Protocol) is an open standard that lets AI tools call external services - databases, APIs, memory stores - through one common interface. So whether I'm in Claude Code, Cursor, or Codex that day, or asking from my phone on a train, everything talks to the same brain.

The name is stolen, and I stole it with pride. In 1945, Vannevar Bush described a hypothetical desk-sized device he called the memex - a machine where a person would keep all their books, records, and letters, "an enlarged intimate supplement to his memory." Eighty years and one weirdly good embeddings API later, you can actually have one.

One thing memex refuses to be: a chatbot. It never writes answers. When my coding agent asks it something, memex hands back ranked excerpts from my own notes, each with a pointer to where it came from, and the agent on my laptop writes the answer from those. That split is deliberate. The brain remembers; the agent thinks. It also means every answer comes with receipts - I can always click through to the note it was quoted from.

Here's a real moment. Fresh session, me typing:

"When did I move memex off PGLite, and why?"

Claude Code quietly calls memex (simplified):

{
  "method": "tools/call",
  "params": {
    "name": "search",
    "arguments": { "q": "PGLite migration why moved", "k": 5 }
  }
}

And gets back excerpts like this:

{
  "hits": [
    {
      "sourcePath": "/vault/20-projects/memex/session-notes.md",
      "content": "PGLite on EFS lost data on SIGKILL - the move
                  to RDS fixed that class of failure...",
      "score": 29
    }
  ]
}

Two seconds, and I have the decision, the date, and the reason - quoted from a retro note written months ago by a version of me who was sure he'd remember. He never remembers.

The moment it clicked

The payoff isn't any single lookup. It's that sessions stopped starting from zero. My kickoff prompt these days boils down to: "You know where everything is - ask the brain." The agent pulls my infrastructure layout, my past decisions, my conventions, and just gets on with it.

Before memex, a session opened like this: "So, I have this EC2 instance, it runs Docker, the tunnel token is in Secrets Manager... no wait, let me check which account that was."

After memex: "Add a health check to the morning briefing timer." That's it. It already knows what the morning briefing is, which timer fires it, and that the chat ID deliberately lives outside the repo.

Fifteen minutes of throat-clearing per session, gone. Multiply by every session, every day.

What's actually inside

Under the hood memex is a Bun + TypeScript service sitting in front of Postgres with the pgvector extension. Everything I'm about to describe lives in the open repo at github.com/timurgaleev/memex - clone it and you get the whole thing: the memory service, the Terraform that builds the AWS setup, the security gates, the docs. The pipeline is boring on purpose:

Ingest. I decide what gets indexed - a folder, a repo, another source. New sources never add themselves; the ones I've configured get re-swept on a schedule, and re-indexing the same path replaces the old version. The brain's database is the store it searches; anything indexed from my vault or a repo can be re-ingested from there, and the database itself gets backed up like any Postgres.
Chunking. Documents get split into pieces that respect structure - markdown by sections, source code by functions and classes. A sanity gate scores incoming text and quarantines scraper junk before it wastes an embedding.
Embeddings. Each chunk becomes a 1024-dimensional vector via Amazon Titan v2. If "embedding" is a new word: it's the meaning of a piece of text written down as a long row of numbers, so that texts about the same thing end up with similar numbers. That's the trick that lets "why did storage keep dying" find a note that never contains the word "dying". Every vector carries a signature (model + dimensions), so if I ever switch embedding models, memex can spot every stale vector - and, with a flag turned on, re-embed them automatically.
Hybrid search. Every query runs keyword search and vector similarity at the same time, then fuses the rankings. Keyword catches exact identifiers - "t4g.medium", "SIGKILL". Vectors catch meaning - "why did storage keep dying". I tried living with each one alone; each alone misses half of what I actually ask.

That's the base. Above it lives the part I actually show off to people - structured memory. Most of it is opt-in and uses paid model calls, so it's off by default in the repo:

Entity facts. memex pulls subject-predicate-object facts out of notes ("memex → runs-on → EC2 t4g.medium") with confidence scores. Facts can supersede each other, decay over time, and keep an audit trail of why something was forgotten. Ask "what do I know about project X" and you get a compiled summary instead of raw search hits - still model-extracted, which is exactly why every fact drags its confidence score along.
A knowledge graph. Notes, projects, and people are linked with typed edges. "Show me everything connected to the n8n server" is a single graph walk.
A code call graph. When memex indexes a repo, it parses who-calls-what. My agent can ask "what breaks if I change this function" and get an actual blast-radius answer. Do you need all of that? No. Hybrid search alone covers most days. But each layer got added because the previous layer fumbled a question I cared about - and since the whole thing is mine, adding a layer is a weekend project.

The night shift

The part nobody sees is my favorite part. While I sleep, memex works.

A maintenance cycle of about fifteen phases runs in the background, and the important thing about it: these phases are deterministic - plain code, no language model doing any thinking. The only AI they touch is the cheap embedding call, so a night's work costs cents. What the phases actually do, in order of how much I'd miss them:

Keep embeddings fresh. Edited notes get re-embedded when their source is re-swept, and the cycle additionally renews vectors that have grown old (the phase is literally called embed-stale), so search never quietly goes out of date. Unchanged text keeps its existing vector - an edit only pays for the paragraphs that actually moved.
Reconnect the web. If I renamed a project, split a note in two, or linked something new, the connections between notes get reconciled so the knowledge graph follows reality instead of drifting away from it.
Take out the trash. Entries whose source is long gone get flagged as orphans so they can be swept out instead of haunting search results forever.
Check the vital signs. The cycle records a snapshot of the corpus - how many documents, how much of it is embedded, what's pending - so trends are visible over weeks. The actual database backups are managed-Postgres routine, as they should be.

Then comes my favorite ritual: every night at 04:30 memex tests its own memory. It replays a golden set of questions - real questions I've asked before, with known good answers - and records how retrieval did: did the right note come back, and how high did it rank. Each run lands as a row of metrics, so if memory quality ever starts drifting, I see it as a falling number on a graph instead of a growing feeling that "search got worse lately." And I can cap what a run is allowed to spend, because I've learned to distrust anything that can burn money unsupervised at 4am.

When something does need attention - a stalled job, low embedding coverage, a pending migration - a built-in advisor ranks the problems by severity, and for most of them hands me the exact fix command. Less "something looks wrong," more "run this."

And for the curious: there's a second, optional night crew that I keep switched off most of the time. It's a chain of AI passes that reads recent notes and tries to distill them - pulling out atomic claims, grouping them into concepts, even proposing opinions with confidence grades attached, all written into its own separate store. It runs only in quiet hours and uses paid model calls - which is exactly why it's off by default in the public repo. Nice to have; not load-bearing.

I wake up, my phone shows the morning briefing (my own wiring again, a small script on the same server), and the brain has already done its chores. Honestly, it's better at maintenance discipline than I am. That was the point.

The AWS build, and its trade-offs

The infrastructure fits on a napkin, and I fight to keep it that way.

One EC2 instance - a $24/month ARM box. One small RDS Postgres. An EFS volume for runtime state, Secrets Manager for credentials, Bedrock for the AI calls. No Kubernetes, no autoscaling group, no load balancer, no message broker. A personal workload doesn't need an orchestrator; the entire runtime survives a docker compose up -d --build.

All of it is Terraform, sitting in the repo under an MIT license - take it apart, copy it, run your own:

# What `terraform apply` creates (illustrative - the repo's
# terraform is plain resources, not a module):
#   VPC + single EC2 (t4g.medium) + RDS Postgres (db.t4g.micro)
#   + EFS + Secrets Manager + IAM + optional CloudTrail
instance_type = "t4g.medium"   # ARM: half the price of x86
db_instance   = "db.t4g.micro"

A few decisions I'd make again:

No inbound ports. The EC2 security group opens nothing to the internet. Nothing. The only way in is a Cloudflare Tunnel - the sidecar dials out to Cloudflare, and my requests ride back down that connection. Shell access goes through AWS SSM Session Manager, so there isn't even an SSH port. None of this makes the box unhackable. It means the attack surface is a bearer token and Cloudflare's edge instead of an open-port buffet for scanners.

Private from whom? Worth being precise here, because "private" carries a lot of weight in this article. My threat model: SaaS vendors training on my notes, a product shutting down and taking my memory with it, careless token leaks. Not on the list: AWS itself - Bedrock processes my text, that's a trade I make consciously and you should too - and not a nation-state. Draw your own line. The point of self-hosting is that you get to draw it.

Layered damage control. The public access token rotates every morning, so a leaked token stops working within a day at worst - and there are more layers under it. Even with a valid token, public reads come back with note bodies redacted, and write operations aren't reachable from the public surface at all. And since retrieved notes eventually get fed to language models, memex neutralizes common instruction-injection tokens in retrieved excerpts before they reach a model - which lowers the risk from the well-known tricks, nothing more. Mitigations, all of it. But each one answers a specific failure I didn't want to explain to myself later.

Bedrock over API keys. Embeddings come from Titan v2. A small Claude Haiku handles query understanding. An optional Claude Sonnet tier does the deeper synthesis work, off by default. Bedrock is AWS-managed, but the calls are authenticated and scoped through my own account's IAM - no third-party API key floating around with my entire note history behind it.

And the bill, since somebody always asks: fifty-something dollars a month on paper, most of which AWS credits currently eat for me; the steady Bedrock share is coffee money, around thirty cents a day. My one splurge was a $23 afternoon of re-embedding the entire corpus with richer context. Worth it exactly once.

Scar tissue

I'll keep this short, but skipping it entirely would turn the article into marketing. The first version kept its database in an embedded Postgres on EFS - a hard SIGKILL ate my index one evening, and that's how RDS earned its place. I once built a full Cognito login setup before admitting that memex serves programs, not people at login forms - tore it out the same week. And retrieval still misses sometimes, when I ask in words I never wrote down. That's exactly what the nightly self-test is for.

Why build instead of buy

There are real products in this space - memory SaaS, notes apps growing AI features, vector-database startups. I went the other way, for three reasons:

Ownership is the feature. Years of decisions, projects, and half-thoughts is the most personal dataset I have. A memory product that shuts down, changes pricing, or "updates its privacy policy" isn't a tool - it's a landlord.
The interface is a standard. MCP means my memory speaks the same protocol AI tools are adopting across the industry. Betting on a protocol beats betting on any single vendor's plugin ecosystem.
The work is the education. Every hard problem in this project - chunking, ranking fusion, fact supersession, tenant isolation, cost control - keeps showing up in real conversations within months, wherever AI tooling is on the table.

For plenty of people, buying is the right call. If your notes live happily in one app and you trust the vendor, go live your life. But if you're the kind of person who reads Terraform for fun, github.com/timurgaleev/memex is waiting. Expect an afternoon to a day of Terraform, DNS, and reading docs.

How my week actually works now

Back to the Forward Deployed Engineer thing, because memex doesn't work alone. Three tools, three jobs, and my week runs through all of them.

vibekit holds the rules - how my AI agents are allowed to behave, what they must never touch, synced into every tool I use from one repo. vibestack holds the workflows - the repeatable loops I run daily: plan a feature, review a diff, ship a release, investigate a bug. And memex holds the memory. Behavior, workflow, recall. Each would survive the others' deletion, which is my working definition of good boundaries.

A normal day: my phone buzzes at 07:00 with a morning briefing my server composed on its own - calendar, mail, house status. On the train I ask the brain something I half-remember about a side project. Somewhere in a conversation - at work, or around a friend's business - someone describes a process that eats two months a year, and something clicks: I've seen this shape before, and my own stack is where I learned to recognize it. In the evening I open a session on a personal project, the agent asks memex where things stand, and we continue where I left off instead of starting over.

That last part is the real change AI brought to my engineering life. Before, my evenings-and-weekends work was throwaway - too little time, too much re-explaining, projects abandoned at the setup stage. Now the setup is remembered, the workflows are one command, and a one-person R&D department is a realistic thing to run before breakfast. Whatever judgment I have about AI tooling - what's actually worth deploying, where it breaks, what it costs - comes from running all of this myself, watching it fail on a Tuesday, and fixing it before work.

Takeaways

AI tools without memory waste your time at scale. The re-explaining tax is invisible until you remove it; then it's obscene.
Retrieval and reasoning should be separate. A brain that returns cited evidence works with any agent that speaks the protocol. A chatbot works with nothing but itself.
Small infrastructure is a superpower. One ARM box, one managed Postgres, one tunnel. Boring enough to never think about, which is the whole point of infrastructure.
Privacy is a spectrum you should place yourself on deliberately - decide who you're defending against before you decide what to build or buy.

The repo is at github.com/timurgaleev/memex - Terraform, docs, and every scar described above. If you build your own brain, or already run one, I genuinely want to hear how you solved the parts I got wrong. Eighty years after Vannevar Bush sketched the memex, the desk-sized memory machine is real, it costs less than a gym membership, and it fits in a t4g.medium.

What would yours remember?

ECS vs EKS: When You DON'T Need Kubernetes - A Practical Guide to Choosing AWS Container Services

Timur Galeev — Sun, 04 Jan 2026 16:20:58 +0000

Introduction

You know what? I see teams spinning up Kubernetes clusters for three microservices all the time. Then they spend two months figuring out pods, ingress controllers, and all that magic. And then they pay $70 per month just for three clusters in different regions, not counting the actual servers.

Here's the honest truth: Kubernetes is a powerful tool but you don't always need it. Amazon ECS is a simpler alternative that handles most tasks faster and cheaper.

In this article I'll show you:

When ECS beats EKS (and saves you tons of money)
Real scenarios with numbers and examples
Ready-to-use code snippets for deploying to both platforms
How to make the decision without headaches

Let's dive in!

Quick Comparison: ECS vs EKS

First let's look at the main differences in a simple table:

Feature	AWS ECS	AWS EKS
Cluster Cost	$0	$0.10/hour (~$70/month)
Setup Complexity	Low (2-4 hours)	High (1-2 days)
Learning Curve	Few days	Several weeks
Management	AWS Console/CLI	kubectl + AWS Console
Ecosystem	AWS services	Entire Kubernetes world
Portability	AWS only	Any cloud/on-prem
Updates	Automatic	Manual (control plane)
Best For	1-10 services	10-100+ services

Architecture: How It Works

ECS Architecture:

Your Application
    ↓
Docker Image (you need this!)
    ↓
Task Definition (container description)
    ↓
ECS Service (manages launch)
    ↓
EC2 or Fargate (where it runs)
    ↓
Container running

EKS Architecture:

Your Application
    ↓
Docker Image
    ↓
Kubernetes Pod specification
    ↓
Deployment/StatefulSet
    ↓
Kubernetes Control Plane ($$$)
    ↓
Worker Nodes
    ↓
Container in Pod

See the difference? ECS has two fewer steps and each one is easier to understand.

When ECS is Your Best Choice

This is where it gets interesting. Many people think Kubernetes is always needed but that's not true. Let's break down real situations where ECS wins.

Scenario 1: Multi-Regional Deployment (3-5 Services)

Imagine: you have a simple API and a couple supporting services. You need to deploy them in three regions - Europe, Asia, USA. For redundancy, you know.

With EKS you pay:

Europe cluster: $70/month
Asia cluster: $70/month
USA cluster: $70/month
Total: $210/month just for the right to run containers

With ECS you pay:

Cluster is free: $0
Total: $0 for management

In other words, save $2,520 per year just on the control plane! And you still gotta pay for the actual servers.

Real Example

I had a project - e-commerce backend. Five services:

API Gateway (Node.js)
Order Service (Python)
Payment Service (Go)
Notification Service (Node.js)
Analytics Worker (Python)

Each service needed a Docker image. Here's a simple Dockerfile example for the Node.js API:

# Dockerfile for API Gateway
FROM node:18-alpine

WORKDIR /app

COPY package*.json ./
RUN npm install --production

COPY . .

EXPOSE 3000
CMD ["node", "server.js"]

We deployed across three regions using ECS Fargate. Setup time: 4 hours including Terraform code. If we'd done it with EKS - that's minimum a week with Helm charts, ingress controllers and all that kitchen.

Here's how we defined the task in ECS (simplified):

# ECS Task Definition - just the container part
resource "aws_ecs_task_definition" "api_gateway" {
  family                   = "api-gateway"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu                      = "256"
  memory                   = "512"

  container_definitions = jsonencode([{
    name      = "api"
    image     = "123456789.dkr.ecr.us-east-1.amazonaws.com/api-gateway:latest"
    essential = true

    portMappings = [{
      containerPort = 3000
      protocol      = "tcp"
    }]

    environment = [
      { name = "NODE_ENV", value = "production" },
      { name = "PORT", value = "3000" }
    ]
  }])
}

Compare this to Kubernetes - you'd need Deployment YAML, Service YAML, maybe Ingress, ConfigMaps... it adds up.

Scenario 2: Quick Start and Simplicity

You're a startup. You have an MVP that needs to ship yesterday. Team of three people nobody knows Kubernetes deeply.

ECS gives you:

Launch in couple hours (not days!)
AWS integration out of the box
No need to hire Kubernetes expert
Less moving parts = less things break

Look I'm not saying Kubernetes is bad. It's awesome! But do you need it when you just wanna run a container? It's like buying a truck to get bread from the store.

Time to learn:

ECS: 2-3 days to work comfortably
EKS: 2-3 weeks minimum (or even a month)

Here's a complete minimal ECS setup with Terraform:

# Minimal ECS cluster
resource "aws_ecs_cluster" "main" {
  name = "my-app-cluster"
}

# ECS Service - runs 2 copies of your container
resource "aws_ecs_service" "app" {
  name            = "my-app"
  cluster         = aws_ecs_cluster.main.id
  task_definition = aws_ecs_task_definition.api_gateway.arn
  desired_count   = 2
  launch_type     = "FARGATE"

  network_configuration {
    subnets          = ["subnet-xxx", "subnet-yyy"]
    security_groups  = ["sg-xxx"]
    assign_public_ip = true
  }
}

That's it! No Helm, no kubectl, no YAML soup.

Scenario 3: AWS-Native Project

Your project is fully in AWS:

Database - RDS
Files - S3
Queues - SQS
Cache - ElastiCache
Logs - CloudWatch

Why Kubernetes here? ECS integrates with these services natively and simpler.

Example - S3 access:

ECS Task Role (simple):

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": "s3:*",
    "Resource": "arn:aws:s3:::my-bucket/*"
  }]
}

Attach the role to Task Definition - done.

In EKS you do the same through IRSA (IAM Roles for Service Accounts):

Setup OIDC provider
Create ServiceAccount in Kubernetes
Link with IAM role
Annotate the pod

More steps = more places to mess up.

When EKS Becomes Necessary

Alright enough praising ECS. Let's be honest - there are situations where EKS is really better.

Scenario 1: Large Microservices Architecture (20+ Services)

When you have 20, 30, 50 microservices - that's different math.

Why EKS wins:

$70 per cluster is fixed price (whether 5 services or 50)
Kubernetes scales complexity better
Ecosystem: Helm, Operators, service mesh (Istio, Linkerd)
Centralized management of all services

Cost example:

With 30 services in one region:

ECS: 30 separate ECS Services = lots of config hard to manage
EKS: One cluster all services in namespaces manage through GitOps

Here $70/month pays for convenience.

A typical Kubernetes deployment:

# Kubernetes Deployment - simpler at scale
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-gateway
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: api-gateway
  template:
    metadata:
      labels:
        app: api-gateway
    spec:
      containers:
      - name: api
        image: my-registry/api-gateway:v1.2.3
        ports:
        - containerPort: 3000
        resources:
          requests:
            memory: "512Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "250m"

With Kubernetes you get built-in health checks, rolling updates, easy rollbacks.

Scenario 2: Multi-Cloud or Hybrid Infrastructure

Your company wants:

Work in AWS and GCP simultaneously
Keep some workloads on-premise
Have ability to migrate between clouds

EKS (Kubernetes) gives portability:

Same YAML manifests work everywhere
Can move applications between clouds
Standardization across all infra

ECS is AWS only. Can't move it anywhere. (ECS anywhere?! :) )

Scenario 3: Advanced Features

GPU workloads for ML/AI: EKS supports GPU nodes out of the box + all tooling like Kubeflow.

Complex networking policies: Network Policies in Kubernetes give precise traffic control between pods.

# Network Policy example
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-policy
spec:
  podSelector:
    matchLabels:
      app: api-gateway
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: frontend
    ports:
    - protocol: TCP
      port: 3000

Stateful applications: StatefulSets Persistent Volumes - all this works better in Kubernetes.

Practical Deployment Examples

Enough theory let's get hands dirty. I'll show how to deploy a simple application to both ECS and EKS. Same application to compare.

Our application: Nginx + simple Node.js API (both need Docker images)

Building Docker Images First

Before deploying anywhere you need Docker images. Here's our setup:

# Dockerfile for our Node.js app
FROM node:18-alpine

WORKDIR /app
COPY package*.json ./
RUN npm ci --production
COPY . .

EXPOSE 3000
CMD ["node", "index.js"]

Build and push:

# Build image
docker build -t my-app:latest .

# Tag for ECR
docker tag my-app:latest 123456789.dkr.ecr.us-east-1.amazonaws.com/my-app:latest

# Push to ECR
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 123456789.dkr.ecr.us-east-1.amazonaws.com
docker push 123456789.dkr.ecr.us-east-1.amazonaws.com/my-app:latest

ECS Deployment with Terraform

Let's start with the simpler one - ECS.

Step 1: VPC Setup

# Create VPC for containers
resource "aws_vpc" "main" {
  cidr_block           = "10.0.0.0/16"
  enable_dns_hostnames = true
  enable_dns_support   = true
}

# Public subnets
resource "aws_subnet" "public" {
  count                   = 2
  vpc_id                  = aws_vpc.main.id
  cidr_block              = "10.0.${count.index}.0/24"
  availability_zone       = data.aws_availability_zones.available.names[count.index]
  map_public_ip_on_launch = true
}

# Internet Gateway
resource "aws_internet_gateway" "main" {
  vpc_id = aws_vpc.main.id
}

Step 2: ECS Cluster and Service

# Create ECS cluster
resource "aws_ecs_cluster" "main" {
  name = "my-app-cluster"
}

# Task Definition - describes your Docker container
resource "aws_ecs_task_definition" "app" {
  family                   = "my-app"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu                      = "256"
  memory                   = "512"

  execution_role_arn = aws_iam_role.ecs_execution.arn
  task_role_arn      = aws_iam_role.ecs_task.arn

  container_definitions = jsonencode([{
    name      = "app"
    image     = "123456789.dkr.ecr.us-east-1.amazonaws.com/my-app:latest"
    essential = true

    portMappings = [{
      containerPort = 3000
      protocol      = "tcp"
    }]

    logConfiguration = {
      logDriver = "awslogs"
      options = {
        "awslogs-group"         = "/ecs/my-app"
        "awslogs-region"        = "us-east-1"
        "awslogs-stream-prefix" = "app"
      }
    }
  }])
}

# ECS Service - runs and maintains containers
resource "aws_ecs_service" "app" {
  name            = "my-app-service"
  cluster         = aws_ecs_cluster.main.id
  task_definition = aws_ecs_task_definition.app.arn
  desired_count   = 2
  launch_type     = "FARGATE"

  network_configuration {
    subnets          = aws_subnet.public[*].id
    security_groups  = [aws_security_group.ecs_tasks.id]
    assign_public_ip = true
  }
}

Step 3: IAM Roles

# Role for ECS to pull Docker images and write logs
resource "aws_iam_role" "ecs_execution" {
  name = "ecs-execution-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "ecs-tasks.amazonaws.com"
      }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "ecs_execution_policy" {
  role       = aws_iam_role.ecs_execution.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
}

# Role for your application (e.g., S3 access)
resource "aws_iam_role" "ecs_task" {
  name = "ecs-task-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "ecs-tasks.amazonaws.com"
      }
    }]
  })
}

Deploy It

terraform init
terraform plan
terraform apply

Done! Container is running.

EKS Deployment with Terraform

Now the same thing but in EKS.

Step 1: EKS Cluster

# EKS cluster
resource "aws_eks_cluster" "main" {
  name     = "my-eks-cluster"
  role_arn = aws_iam_role.eks_cluster.arn
  version  = "1.28"

  vpc_config {
    subnet_ids = concat(aws_subnet.public[*].id, aws_subnet.private[*].id)
  }
}

# Worker nodes
resource "aws_eks_node_group" "main" {
  cluster_name    = aws_eks_cluster.main.name
  node_group_name = "main-nodes"
  node_role_arn   = aws_iam_role.eks_node.arn
  subnet_ids      = aws_subnet.private[*].id

  scaling_config {
    desired_size = 2
    max_size     = 4
    min_size     = 1
  }

  instance_types = ["t3.medium"]
}

Step 2: IAM for EKS

# Cluster role
resource "aws_iam_role" "eks_cluster" {
  name = "eks-cluster-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "eks.amazonaws.com"
      }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "eks_cluster_policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.eks_cluster.name
}

# Node role
resource "aws_iam_role" "eks_node" {
  name = "eks-node-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "ec2.amazonaws.com"
      }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "eks_worker_node" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"
  role       = aws_iam_role.eks_node.name
}

resource "aws_iam_role_policy_attachment" "eks_cni" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
  role       = aws_iam_role.eks_node.name
}

Step 3: Kubernetes Manifests

After cluster is created deploy application with kubectl:

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: app
        image: 123456789.dkr.ecr.us-east-1.amazonaws.com/my-app:latest
        ports:
        - containerPort: 3000
        resources:
          requests:
            memory: "512Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "250m"
---
apiVersion: v1
kind: Service
metadata:
  name: my-app-service
spec:
  type: LoadBalancer
  selector:
    app: my-app
  ports:
  - port: 80
    targetPort: 3000

Deploy It

# 1. Apply Terraform
terraform init
terraform apply

# 2. Configure kubectl
aws eks update-kubeconfig --name my-eks-cluster --region us-east-1

# 3. Check nodes
kubectl get nodes

# 4. Deploy application
kubectl apply -f deployment.yaml

# 5. Check status
kubectl get pods
kubectl get svc

Difference:

ECS: one terraform apply and done
EKS: terraform apply + kubectl commands + wait for everything to come up

Complexity Comparison

Action	ECS	EKS
Config files	3-4 Terraform files	4-5 Terraform + YAML manifests
First deploy time	5-7 minutes	15-20 minutes
Commands to run	2 (init apply)	5+ (terraform + kubectl)
Need to know	AWS Terraform Docker	AWS Terraform Kubernetes kubectl Docker

Real Cases and Economics

Let's calculate concrete numbers for typical scenarios.

Case 1: Startup with 5 Microservices in 3 Regions

Requirements:

5 services (API Workers Background Jobs)
3 regions: US EU Asia
2 instances each service
All need Docker images built and stored in ECR

ECS Fargate:

Cluster cost: $0
ECR storage: ~$5/month (for Docker images)
Compute (Fargate):
  - 5 services × 2 instances × 3 regions = 30 tasks
  - Each task: 0.25 vCPU 512 MB
  - $0.04048/hour per vCPU $0.004445/hour per GB
  - (~0.25 × $0.04048 + 0.5 × $0.004445) × 730 hours = ~$9/task/month
  - 30 tasks × $9 = $270/month

Total: ~$275/month

EKS:

Cluster cost: $70 × 3 regions = $210/month
ECR storage: ~$5/month (same Docker images)
Compute (EC2 nodes):
  - Minimum 2× t3.medium per region = 6 instances
  - t3.medium = $0.0416/hour × 730 = ~$30/month
  - 6 × $30 = $180/month

Total: $210 + $5 + $180 = $395/month

Savings with ECS: $120/month or $1440/year

Plus with ECS you don't pay DevOps engineer to manage Kubernetes :)

Case 2: Large Project with 30 Services in 1 Region

ECS:

Cluster: $0
Management: 30 separate ECS Services (hard to manage!)
Compute: depends on load

EKS:

Cluster: $70/month
Management: One namespace GitOps Helm (easier!)
Compute: same + better resource utilization

Here EKS wins on management convenience. $70 pays for itself.

Time for Setup and Maintenance

Real numbers from my experience:

Initial setup:

ECS: 4 hours (Terraform + tests + Docker builds)
EKS: 2 days (cluster + addons + monitoring setup + Docker builds)

Weekly maintenance:

ECS: ~30 minutes (check logs updates)
EKS: ~2 hours (updates cluster checks monitoring)

Platform updates:

ECS: automatic
EKS: need to update control plane once a year (takes half a day with tests)

Decision Checklist: What to Choose?

So here's a simple flowchart for decision making:

Choose ECS if:

✅ You have less than 10-15 microservices
✅ Project is AWS only (no multi-cloud plans)
✅ Team doesn't know Kubernetes (and doesn't want to learn)
✅ Need to launch quickly (MVP startup)
✅ Budget is limited
✅ Simple application without complex dependencies
✅ Multi-regional deploy (save on clusters)
✅ Comfortable with Docker basics

Choose EKS if:

✅ More than 20+ microservices
✅ Need portability (multi-cloud hybrid)
✅ Team knows Kubernetes
✅ Need advanced features (service mesh operators)
✅ GPU workloads for ML/AI
✅ Already using Kubernetes elsewhere
✅ Complex microservices architecture
✅ Want access to Kubernetes ecosystem

Middle Ground

You can start with ECS and migrate later!

Many companies do this:

Start on ECS (fast and cheap)
Grow to 15-20 services
Kubernetes developers join the team
Gradually migrate to EKS

This is normal evolution. Don't go Kubernetes just "because it's cool".

Conclusions

Here's what's important to remember:

ECS is not a "second-rate" option. It's a full-fledged solution that handles many tasks excellently. Yes EKS is more powerful in capabilities but most projects simply don't need those capabilities.

Main ECS advantages:

Free control plane (save $70-210+ per month)
Simplicity and launch speed
Less operational overhead
Native AWS integration
Perfect for multi-regional deployment of small services

When EKS is really needed:

Large scale (20+ services)
Code portability
Advanced Kubernetes features
Already have expertise in team

My advice: Don't chase the hype. Start with ECS if the task allows. Save time money and nerves. And when you really grow into Kubernetes - then migrate.

Kubernetes is like a Ferrari - cool car but for a trip to the store a regular Toyota works fine. And uses less gas 😄

Final Thoughts

The choice between ECS and EKS isn't about "better" or "worse" - it's about right tool for the job.

Start simple. ECS lets you ship fast without the Kubernetes learning curve. Your Docker skills transfer directly. AWS handles the orchestration.

As you grow, reassess. When you hit 15-20 services or need multi-cloud, EKS makes sense. But many successful companies run production on ECS for years.

Remember: Complexity is a cost. Every abstraction layer you add costs time money and mental overhead. Sometimes the best architecture is the simplest one that works.

Both platforms use Docker. Both run containers. Both scale. The question is: how much complexity do you actually need?

Choose wisely!

Sources

Building an AI-Optimized Platform on Amazon EKS with NVIDIA NIM and OpenAI Models

Timur Galeev — Wed, 18 Dec 2024 22:46:10 +0000

Introduction

The rise of artificial intelligence (AI) has brought about an unprecedented demand for infrastructure that can handle large-scale computations, support GPU acceleration, and provide scalable, flexible management of workloads. Kubernetes has emerged as a leading platform for orchestrating these workloads, and Amazon Elastic Kubernetes Service (EKS) extends Kubernetes’ capabilities by simplifying deployment and scaling in the cloud.

NVIDIA Infrastructure Manager (NIM) complements Kubernetes by optimizing GPU workloads, a critical need for training large language models (LLMs), computer vision, and other computationally intensive AI tasks. Additionally, OpenAI models can be integrated into this ecosystem to unlock cutting-edge AI capabilities, such as text generation, image recognition, and decision-making systems.

This article provides an in-depth guide to building a complete AI platform using EKS, NVIDIA NIM, and OpenAI models, with Terraform automating the deployment. Whether you are an AI researcher or a business looking to adopt AI, this guide outlines how to build a robust and scalable platform. Complete code for this setup is available on GitHub https://github.com/timurgaleev/eks-nim-llm-openai.

Why Choose NVIDIA NIM and EKS for AI Workloads?

Challenges of AI Workloads

AI applications, especially those involving LLMs, have unique challenges:

GPU Resource Management: Training and inference rely on GPUs, which are scarce and expensive resources. Efficient allocation and monitoring are crucial.

Scalability: AI workloads often need to scale dynamically based on user demand or data processing requirements.

Storage for Large Datasets: AI models and datasets can require hundreds of gigabytes, necessitating persistent, shared, and scalable storage.

Observability: Monitoring system performance, especially GPU utilization and latency, is essential for optimizing workloads.

NVIDIA NIM: A Solution for GPU Workloads

NVIDIA NIM addresses these challenges by providing:

GPU Scheduling: Maximizes GPU usage across workloads.

Integration with Kubernetes: Leverages Kubernetes to manage pods, jobs, and resources efficiently.

AI Model Management: Simplifies deployment and scaling of AI models with Helm charts and Kubernetes CRDs (Custom Resource Definitions).

Support for Persistent Storage: Integrates with shared storage solutions like AWS EFS for storing datasets and models.

Amazon EKS: A Scalable Kubernetes Solution

Amazon EKS adds value by:

Managed Kubernetes: Reduces operational overhead by handling Kubernetes cluster setup, updates, and management.

Elastic Compute Integration: Dynamically provisions GPU-enabled instances, such as g4dn and p4d, to handle AI workloads. Ensure that your AWS account has sufficient quotas and availability for these instance types to avoid provisioning issues.

Built-in Security: Integrates with AWS IAM and VPC for secure access and network segmentation.

Together, NVIDIA NIM and Amazon EKS create a powerful platform for AI model training, inference, and experimentation.

Architecture Overview

The platform architecture integrates NVIDIA NIM and OpenAI models into an EKS cluster, combining compute, storage, and monitoring components.

Key Components

EKS Cluster: Manages Kubernetes workloads and scales GPU-enabled nodes.

Karpenter: Dynamically provisions and scales nodes (CPU and GPU) based on workload demands, optimizing resource utilization and cost.

GPU Node Groups: Nodes equipped with NVIDIA GPUs for ML and AI inference tasks.

NVIDIA NIM: Deploys GPU workloads, manages AI pipelines, and integrates with Kubernetes.

OpenAI Web UI: Provides a user-friendly interface for interacting with AI models.

Persistent Storage: AWS EFS supports shared storage for datasets and models.

Observability Tools: Prometheus and Grafana offer real-time monitoring of system metrics, including GPU utilization and pod performance.

Deployment Guide

This guide provides step-by-step instructions to deploy the architecture using Terraform. While the focus is on essential components like EKS, GPU workloads, and observability, we skip detailed VPC configuration to allow flexibility based on your specific requirements.

For a VPC example that fits this deployment, refer to the repository: https://github.com/timurgaleev/eks-nim-llm-openai.

Step 1: Provisioning the EKS Cluster

Provisioning an Amazon EKS cluster is the foundation for Kubernetes workloads. Below is the EKS Cluster Configuration with key highlights to focus on scalability, system add-ons, and Karpenter integration.

EKS Cluster Configuration

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 19.15"

  cluster_name                   = local.name
  cluster_version                = var.eks_cluster_version
  cluster_endpoint_public_access = true

  vpc_id     = module.vpc.vpc_id
  subnet_ids = compact([
    for subnet_id, cidr_block in zipmap(module.vpc.private_subnets, module.vpc.private_subnets_cidr_blocks) :
    substr(cidr_block, 0, 4) == "100." ? subnet_id : null
  ])

  manage_aws_auth_configmap = true
  aws_auth_roles = [
    {
      rolearn  = module.eks_blueprints_addons.karpenter.node_iam_role_arn
      username = "system:node:{{EC2PrivateDNSName}}"
      groups = [
        "system:bootstrappers",
        "system:nodes"
      ]
    }
  ]

  eks_managed_node_group_defaults = {
    iam_role_additional_policies = {
      AmazonSSMManagedInstanceCore = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
    }
    ebs_optimized = true
    block_device_mappings = {
      xvda = {
        device_name = "/dev/xvda"
        ebs = {
          volume_size = 100
          volume_type = "gp3"
        }
      }
    }
  }

  eks_managed_node_groups = {
    core_node_group = {
      name            = "core-node-group"
      description     = "EKS Core node group for hosting system add-ons"
      subnet_ids      = compact([
        for subnet_id, cidr_block in zipmap(module.vpc.private_subnets, module.vpc.private_subnets_cidr_blocks) :
        substr(cidr_block, 0, 4) == "100." ? subnet_id : null
      ])
      ami_type        = "AL2_x86_64"
      instance_types  = ["m5.xlarge"]
      capacity_type   = "SPOT"
      desired_size    = 2
      min_size        = 2
      max_size        = 4
      labels = {
        WorkerType    = "SPOT"
        NodeGroupType = "core"
      }
      tags = merge(local.tags, { Name = "core-node-grp" })
    }
  }
}

Key Highlights

1. Networking:

Subnets are filtered to include only CIDR blocks starting with 100. to ensure specific subnet assignment for nodes.

2. IAM and Auth:

Integration with Karpenter is configured via the aws_auth_rolesblock, allowing Karpenter to dynamically provision nodes.

3. Managed Node Groups:

Core Node Group:
- Optimized for system-level workloads.
- Configured with m5.xlarge spot instances for cost efficiency.
- Labels such as NodeGroupType: core and taints can be used to restrict workloads to this node group.

4. Storage:

Nodes are configured with gp3 root volumes (100 GiB) for system usage. Additional storage for workloads should be configured separately.

5. Scaling:

Use Karpenter for workload-based scaling instead of additional managed node groups. The eks_managed_node_groups block here is only for critical system workloads.

Step 2: Deploying NVIDIA NIM for AI Workloads

Deploying NVIDIA NIM (NVIDIA Inference Manager) requires configuring persistent storage for large datasets and allocating GPU resources for optimal performance. Here's an expanded guide breaking down the essential steps.

1. Persistent Storage with AWS EFS

AI workloads often require storage that exceeds local node capacity. AWS EFS (Elastic File System) provides a shared and scalable storage solution across multiple pods. Below is the configuration for creating a Persistent Volume Claim (PVC) backed by EFS:

Code: Persistent Volume Claim (PVC)

kubernetes_persistent_volume_claim_v1 "efs_pvc" {
  metadata {
    name      = "efs-storage"
    namespace = "nim"
  }
  spec {
    access_modes       = ["ReadWriteMany"] # Enables sharing storage across multiple pods.
    storage_class_name = "efs"             # Links the PVC to an EFS storage class.
    resources {
      requests = {
        storage = "200Gi" # Reserves 200 GiB of scalable storage.
      }
    }
  }
}

Key Points:

Access Mode: "ReadWriteMany" allows simultaneous access by multiple pods, critical for parallel workloads.
Storage Class: Must correspond to an EFS provisioner configured in the Kubernetes cluster.
Capacity: Start with 200 GiB and scale as per your dataset requirements.

2. Deploying NVIDIA NIM Helm Chart

After configuring storage, deploy NVIDIA NIM using Helm. The Helm chart simplifies GPU allocation and links the persistent storage to NIM-managed workloads.

Configure the NGC API Key

Before deploying NVIDIA NIM, you need to retrieve your NGC API Key from NVIDIA’s cloud platform and set it as an environment variable. This key enables secure authentication with NVIDIA’s container registry and services.

Steps to Retrieve the NGC API Key:

Log in to your NGC account.
Navigate to Setup > API Keys.
Click Generate API Key if you don’t already have one.
Copy the generated key to use in your deployment process.

Set the NGC API Key as an Environment Variable:

Run the following command in your terminal to make the key accessible to Terraform during deployment:

export TF_VAR_ngc_api_key=<replace-me>

Replace <replace-me> with your actual API key. This key will be passed to NVIDIA NIM to enable seamless model deployment.

Code: Helm Release for NVIDIA NIM

helm_release "nim_llm" {
  name      = "nim-llm"
  chart     = "./nim-llm"                # Points to the NIM Helm chart location.
  namespace = "nim"
  values = [
    templatefile("nim-llm-values.yaml", {
      model_id    = var.model_id            # Specifies the LLM model (e.g., GPT-like models).
      num_gpu     = var.num_gpu             # Allocates GPU resources for inference tasks.
      ngc_api_key = var.ngc_api_key
      pvc_name    = kubernetes_persistent_volume_claim_v1.efs_pvc.metadata[0].name
    })
  ]
}

Key Points:

model_id: The identifier of the model being deployed (e.g., GPT-3, BERT).
num_gpu: Configures GPU resources for inference tasks. The value should align with the instance type used in your cluster (e.g., g4dn.xlarge for one GPU).
pvc_name: Links the EFS-backed PVC to the workload for storing large datasets or models.

3. Configuration Highlights

Why Persistent Storage?

AI models and datasets are often larger than the node's local storage. Using EFS ensures:
- Scalability: Adjust storage as required without downtime.
- High Availability: Accessible across multiple Availability Zones.

GPU Allocation

NVIDIA NIM optimizes GPU usage for inference. Use the num_gpu variable to specify the number of GPUs for your workload, ensuring efficient resource utilization.

Summary

Storage Configuration: Use AWS EFS with Kubernetes PVC for shared, scalable storage across pods.
GPU Allocation: NVIDIA NIM enables efficient GPU resource management for AI inference tasks.
Helm Chart Deployment: Leverage Helm for streamlined deployment, linking GPU resources and persistent storage.

Step 3: Adding OpenAI Web UI

The OpenAI Web UI provides an interface for users to interact with deployed AI models.

"helm_release" "openai_webui" {
  name       = "openai-webui"
  chart      = "open-webui"
  repository = "https://helm.openwebui.com/"
  namespace  = "openai-webui"
  values = [
    jsonencode({
      replicaCount = 1,
      image = {
        repository = "ghcr.io/open-webui/open-webui"
        tag        = "main"
      }
    })
  ]
}

Step 4: Observability with Prometheus, Grafana, and Custom Metrics

Prometheus and Grafana are essential tools for monitoring AI workloads. Prometheus collects resource metrics, including GPU-specific data, while Grafana visualizes these metrics through tailored dashboards. These tools help ensure that AI operations are running smoothly and efficiently.

To extend observability, the Prometheus Adapter is configured with custom rules for tracking AI-specific metrics. Key configurations include:

Tracking Active Requests: Using the num_requests_running metric, Prometheus monitors the number of ongoing requests, providing insights into workload intensity.
Inference Queue Monitoring: The nv_inference_queue_duration_us metric tracks NVIDIA inference queue times, converted into milliseconds for enhanced readability.

Sample Configuration for Prometheus Adapter:

prometheus:
  url: http://kube-prometheus-stack-prometheus.${prometheus_namespace}
  port: 9090
rules:
  default: false
  custom:
  - seriesQuery: '{__name__=~"num_requests_running"}'
    resources:
      template: <<.Resource>>
    name:
      matches: "num_requests_running"
      as: ""
    metricsQuery: sum(<<.Series>>{<<.LabelMatchers>>}) by (<<.GroupBy>>)
  - seriesQuery: 'nv_inference_queue_duration_us{namespace!="", pod!=""}'
    resources:
      overrides:
        namespace:
          resource: "namespace"
        pod:
          resource: "pod"
    name:
      matches: "nv_inference_queue_duration_us"
      as: "nv_inference_queue_duration_ms"
    metricsQuery: 'avg(rate(nv_inference_queue_duration_us{<<.LabelMatchers>>}[1m])/1000) by (<<.GroupBy>>)'

These configurations enable Prometheus to expose meaningful custom metrics that are critical for scaling and optimizing AI workloads. By integrating these metrics into Grafana dashboards, users gain actionable insights into system performance and bottlenecks.

Step 5: Scaling and Optimization with Karpenter

In large-scale AI deployments, workload demands fluctuate significantly. Dynamic scaling is essential for managing these workloads effectively while minimizing costs. Karpenter, a Kubernetes-native cluster autoscaler, provides powerful mechanisms for optimizing resource utilization. It dynamically provisions nodes tailored to the specific demands of applications, including GPU-heavy AI workloads.

This section integrates Karpenter into the EKS Blueprint framework, highlighting its configuration for both CPU and GPU workloads. The full implementation and configurations are available in the https://github.com/timurgaleev/eks-nim-llm-openai.

Deploying Karpenter with EKS Blueprints

Karpenter is added to the EKS cluster as a Blueprint add-on. Below is an example of the configuration block for enabling Karpenter, focusing on both CPU and GPU workload optimization:

module "eks_blueprints_addons" {
  source  = "aws-ia/eks-blueprints-addons/aws"
  version = "~> 1.2"

  enable_karpenter                  = true
  karpenter_enable_spot_termination = true
  karpenter_node = {
    iam_role_additional_policies = {
      AmazonSSMManagedInstanceCore = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
    }
  }
  karpenter = {
    chart_version = "0.37.0"
  }
}

This configuration enables Karpenter with support for Spot instance termination handling and assigns additional IAM policies for managing nodes.

Configuring Karpenter for CPU and GPU Workloads

For effective scaling, Karpenter relies on Provisioner configurations tailored to workload requirements. The following examples showcase how Karpenter dynamically provisions CPU and GPU nodes.

CPU Workloads

name: cpu-karpenter
clusterName: ${module.eks.cluster_name}
ec2NodeClass:
  karpenterRole: ${split("/", module.eks_blueprints_addons.karpenter.node_iam_role_arn)[1]}
  subnetSelectorTerms:
    id: ${module.vpc.private_subnets[2]}
  securityGroupSelectorTerms:
    tags:
      Name: ${module.eks.cluster_name}-node
  instanceStorePolicy: RAID0

nodePool:
  labels:
    - type: karpenter
    - NodeGroupType: cpu-karpenter
  requirements:
    - key: "karpenter.k8s.aws/instance-family"
      operator: In
      values: ["m5"]
    - key: "karpenter.k8s.aws/instance-size"
      operator: In
      values: ["xlarge", "2xlarge", "4xlarge"]
    - key: "kubernetes.io/arch"
      operator: In
      values: ["amd64"]
    - key: "karpenter.sh/capacity-type"
      operator: In
      values: ["spot", "on-demand"]
  limits:
    cpu: 1000
  disruption:
    consolidationPolicy: WhenEmpty
    consolidateAfter: 180s
    expireAfter: 720h
  weight: 100

GPU Workloads

name: gpu-workloads
clusterName: ${module.eks.cluster_name}
ec2NodeClass:
  karpenterRole: ${split("/", module.eks_blueprints_addons.karpenter.node_iam_role_arn)[1]}
  subnetSelectorTerms:
    id: ${module.vpc.private_subnets[1]}
  securityGroupSelectorTerms:
    tags:
      Name: ${module.eks.cluster_name}-node
  instanceStorePolicy: RAID0

nodePool:
  labels:
    - type: karpenter
    - NodeGroupType: gpu-workloads
  requirements:
    - key: "karpenter.k8s.aws/instance-family"
      operator: In
      values: ["g5", "p4", "p5"]  # GPU instances
    - key: "karpenter.k8s.aws/instance-size"
      operator: In
      values: ["2xlarge", "4xlarge", "8xlarge", "12xlarge"]
    - key: "kubernetes.io/arch"
      operator: In
      values: ["amd64"]
    - key: "karpenter.sh/capacity-type"
      operator: In
      values: ["spot", "on-demand"]
  limits:
    cpu: 1000
  disruption:
    consolidationPolicy: WhenEmpty
    consolidateAfter: 180s
    expireAfter: 720h
  weight: 100

Terraform Automation Scripts

To streamline the deployment and teardown of resources, the project includes two utility scripts: install.sh and cleanup.sh.

install.sh: Automates the deployment process. It initializes Terraform, applies modules sequentially (e.g., VPC and EKS), and ensures all resources are provisioned successfully. A final Terraform apply captures any remaining dependencies.

cleanup.sh: Safely destroys the deployed infrastructure. It handles dependencies like Kubernetes services, Load Balancers, and Security Groups, ensuring proper teardown order. Each module is destroyed sequentially, with a final pass to catch residual resources.

These scripts enhance operational efficiency and minimize errors during deployment and cleanup phases, making the workflow more robust and reproducible.

Key Features of Karpenter in AI Ecosystems

Dynamic Node Provisioning: Automatically provisions CPU or GPU nodes based on real-time workload needs.
Cost Optimization: Leverages Spot instances while ensuring reliable on-demand scaling for critical workloads.
Enhanced Resource Utilization: Consolidates underutilized nodes and removes idle resources with disruption policies.
Tailored Scaling Policies: Supports node pools for diverse workload types, such as inference tasks or data preprocessing.

Karpenter’s integration with GPU-optimized workloads ensures that demanding AI models benefit from high-performance compute nodes while maintaining cost efficiency.

Use Cases

1. AI Model Training

NVIDIA NIM’s GPU optimizations allow for efficient training of models like BERT or GPT, reducing runtime and costs.

2. Real-Time Inference

Deploy models for real-time applications such as fraud detection, image recognition, or natural language understanding.

3. Experimentation and Research

With the OpenAI Web UI, data scientists can quickly test and iterate on models.

Conclusion

This platform enables the scalable and efficient deployment of AI workloads by integrating NVIDIA NIM with Amazon EKS. Terraform automates the process, ensuring repeatable and reliable setups. With GPU optimization, persistent storage, and observability tools, the platform is well-suited for businesses and researchers alike.

By following this guide, you can build a scalable and efficient AI platform. For detailed code and further exploration, visit the GitHub repository https://github.com/timurgaleev/eks-nim-llm-openai.

Deploying AWS EKS with Terraform and Blueprints Addons

Timur Galeev — Thu, 07 Nov 2024 09:07:18 +0000

After a pause from covering AWS and infrastructure management, I’m back with insights for those looking to navigate the world of AWS containers and Kubernetes with ease. For anyone new to deploying Kubernetes in AWS, leveraging Terraform for setting up an EKS (Elastic Kubernetes Service) cluster can be a game-changer. By combining Terraform’s infrastructure-as-code capabilities with AWS’s EKS Blueprints Addons, users can create a scalable, production-ready Kubernetes environment without the usual complexity.

In this article, I'll guide you through using Terraform to deploy EKS with essential add-ons, which streamline the configuration and management of your Kubernetes clusters. With these modular add-ons, you can quickly incorporate features like CoreDNS, the AWS Load Balancer Controller, and other powerful tools to customize and enhance your setup. Whether you’re new to container orchestration or just seeking an efficient AWS solution, this guide will help you build a resilient EKS environment in a few straightforward steps.

So let’s start.

Setting Up the VPC for EKS

The VPC configuration is foundational for your EKS cluster, establishing a secure, isolated environment with both public and private subnets. Private subnets are typically used to host your Kubernetes nodes, keeping them inaccessible from the internet. Here’s the configuration provided in the vpc.tf file, which sets up both public and private subnets along with NAT and Internet Gateway options for flexible networking.

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name                 = local.name
  cidr                 = var.vpc_cidr
  azs                  = local.azs
  secondary_cidr_blocks = var.secondary_cidr_blocks
  private_subnets      = concat(local.private_subnets, local.secondary_ip_range_private_subnets)
  public_subnets       = local.public_subnets
  enable_nat_gateway   = true
  single_nat_gateway   = true
  public_subnet_tags   = {"kubernetes.io/role/elb" = 1}
  private_subnet_tags  = {
    "kubernetes.io/role/internal-elb" = 1
    "karpenter.sh/discovery" = local.name
  }
  tags = local.tags
}

This setup:

Creates private and public subnets across multiple availability zones.
Configures a secondary CIDR block for the EKS data plane, which is crucial for large-scale deployments.
Enables a NAT gateway for private subnets, ensuring secure internet access for internal resources.
Tags subnets for Kubernetes service and discovery, essential for integration with other AWS services like load balancers and Karpenter.

Deploying EKS with Managed Node Groups

Now that the VPC is configured, let’s move on to deploying the EKS cluster with the eks.tf file configuration. This setup includes defining managed node groups within the EKS cluster, specifying node configurations, security rules, and IAM roles.

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 19.15"

  cluster_name                   = local.name
  cluster_version                = var.eks_cluster_version
  cluster_endpoint_public_access = true
  vpc_id                         = module.vpc.vpc_id
  subnet_ids                     = compact([for subnet_id, cidr_block in zipmap(module.vpc.private_subnets, module.vpc.private_subnets_cidr_blocks) : substr(cidr_block, 0, 4) == "100." ? subnet_id : null])

  aws_auth_roles = [
    {
      rolearn  = module.eks_blueprints_addons.karpenter.node_iam_role_arn
      username = "system:node:{{EC2PrivateDNSName}}"
      groups   = ["system:bootstrappers", "system:nodes"]
    }
  ]

  eks_managed_node_groups = {
    core_node_group = {
      name             = "core-node-group"
      ami_type         = "AL2_x86_64"
      min_size         = 2
      max_size         = 8
      desired_size     = 2
      instance_types   = ["m5.xlarge"]
      capacity_type    = "SPOT"
      labels           = { WorkerType = "SPOT", NodeGroupType = "core" }
      tags             = merge(local.tags, { Name = "core-node-grp" })
    }
  }
}

Key components:

VPC and Subnets: The vpc_id and subnet_ids reference the private subnets, providing a secure foundation for EKS nodes.
Managed Node Groups: This setup defines a core node group with spot instances (capacity_type = "SPOT") to optimize cost, with configurable instance types, sizes, and labels for workload placement.
Security Rules and IAM Roles: Configures additional security rules to manage access between nodes and clusters, along with IAM roles to control permissions for Karpenter and node management.

Configuring EKS Add-ons

Add-ons enhance your EKS cluster by integrating additional AWS services and open-source tools. With the EKS Blueprints, you can easily set up these add-ons, which range from storage solutions to observability and monitoring tools.

Setting Up the EBS CSI Driver for Persistent Storage

The Amazon EBS CSI Driver is essential for persistent storage on EKS. This module configures the necessary IAM roles for the driver, enabling it to provision and manage EBS volumes.

module "ebs_csi_driver_irsa" {
  source                = "terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks"
  version               = "~> 5.20"
  role_name_prefix      = format("%s-%s-", local.name, "ebs-csi-driver")
  attach_ebs_csi_policy = true
  oidc_providers = {
    main = {
      provider_arn               = module.eks.oidc_provider_arn
      namespace_service_accounts = ["kube-system:ebs-csi-controller-sa"]
    }
  }
  tags = local.tags
}

This configuration creates an IAM role for the EBS CSI Driver using IAM Roles for Service Accounts (IRSA), which allows the driver to interact with EBS securely.

Enabling Amazon CloudWatch Observability

The amazon-cloudwatch-observability add-on integrates CloudWatch for monitoring and logging, providing insights into your cluster’s performance.

eks_addons = {
  amazon-cloudwatch-observability = {
    preserve                 = true
    service_account_role_arn = aws_iam_role.cloudwatch_observability_role.arn
  }
}

This snippet specifies the IAM role required for CloudWatch, enabling detailed observability for your workloads.

Integrating AWS Load Balancer Controller

The AWS Load Balancer Controller allows you to provision and manage Application Load Balancers (ALBs) for Kubernetes services. Here’s how it’s configured:

enable_aws_load_balancer_controller = true
aws_load_balancer_controller = {
  set = [{
    name  = "enableServiceMutatorWebhook"
    value = "false"
  }]
}

The enableServiceMutatorWebhook setting is disabled to avoid automatic modification of service annotations, making it ideal for custom configurations.

Adding Karpenter for Autoscaling

Karpenter is an open-source autoscaler designed for Kubernetes, enabling efficient and dynamic scaling of EC2 instances based on workload requirements. This configuration sets up Karpenter with support for spot instances, reducing costs for non-critical workloads.

enable_karpenter                  = true
karpenter_enable_spot_termination = true
karpenter_node = {
  iam_role_additional_policies = {
    AmazonSSMManagedInstanceCore = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
  }
}
karpenter = {
  chart_version       = "0.37.0"
  repository_username = data.aws_ecrpublic_authorization_token.token.user_name
  repository_password = data.aws_ecrpublic_authorization_token.token.password
}

This configuration includes additional IAM policies for Karpenter nodes, making it easier to integrate with AWS services like EC2 for flexible scaling.

These add-ons, configured through the AWS EKS Blueprints and Terraform, help streamline Kubernetes management on AWS while offering enhanced storage, observability, and autoscaling.

To explore the complete configuration, you can find the full code in the GitHub repository https://github.com/timurgaleev/aws-eks-terraform-addons. The repository includes install.sh to deploy the EKS cluster and configure the add-ons seamlessly, along with cleanup.sh to tear down the environment when it’s no longer needed.

Conclusion

This Terraform setup provides a powerful framework for deploying EKS with essential add-ons, such as storage, observability, and autoscaling, to support scalable applications. Specifically, this configuration is designed to enable deployment of applications like OpenAI Chat, showcasing Kubernetes' flexibility for real-time, interactive workloads. With this setup, you’re ready to deploy and manage robust, production-grade EKS clusters in AWS.

Beginning the Journey into ML, AI and GenAI on AWS

Timur Galeev — Mon, 22 Jan 2024 22:08:13 +0000

Machine Learning (ML), Artificial Intelligence (AI), and Generative Artificial Intelligence (GenAI) are transformative technologies that have the potential to revolutionize industries across the globe.

At the last AWS re:Invent, there were numerous updates related to ML/AI and everything associated with these technologies. I also decided to delve into these topics and immerse myself in this field.

I won't delve into explaining the meanings of ML, AI, DL(Deep Learning), and GenAI. However, I'd like to touch upon FMs and LLM as we will focus our attention there. I found myself losing the same question when I came across this topic in my reading or listening. :)

Foundational Models (FMs) within the AWS ecosystem represent fundamental structures and algorithms essential for diverse AI applications. These models, often created by industry-leading AI companies, are integral to the development and functionality of AWS services, shaping the landscape of artificial intelligence on the platform. In the context of Amazon Bedrock, Language Models (LMs) play a pivotal role. These LMs contribute to the service's linguistic capabilities, facilitating advanced language understanding and content generation within the AWS environment.

AWS provides various services for Machine Learning and Artificial Intelligence, including Amazon SageMaker, AWS DeepLens, AWS DeepComposer, Amazon Forecast and more. Familiarize yourself with the services available to determine which ones suit your specific needs.

Generative Artificial Intelligence (GenAI) is a type of artificial intelligence that can generate text, images, or other media using generative models. AWS offers a range of services for building and scaling generative AI applications, including Amazon SageMaker, Amazon Rekognition, AWS DeepRacer, and Amazon Forecast. AWS has also invested in developing foundation models (FMs) for generative AI, which are ultra-large machine learning models that generative AI relies on. AWS has also launched the Generative AI Innovation Center, which connects AWS AI and ML experts with customers around the world to help them envision, design, and launch new generative AI products and services. Generative AI has the potential to revolutionize the way we create and consume media, but it is important to use it responsibly and ethically.

Some examples GenAI: One of the most well-known examples of GenAI is ChatGPT, launched by OpenAI, which became wildly popular overnight and galvanized public attention. Another model from OpenAI, called text-embedding-ada-002, is specifically designed to work with embeddings—a type of database specifically designed to feed data into large language models (LLM). However, it’s important to note that generative AI creates artifacts that can be inaccurate or biased, making human validation essential and potentially limiting the time it saves workers. Therefore, end users should be realistic about the value they are looking to achieve, especially when using a service as is.

I've also delved a bit deeper into Broad AI when learning GenAI and I'd like to show this in the form of the following picture as it explains a lot.

Layers from broad artificial Intelligence to generative AI

Broad AI includes task-specific algorithms, Machine Learning (ML), and Deep Learning. These layers enable AI to perform tasks like image recognition, natural language processing, and complex pattern modeling.

The transition to GenAI involves Transfer Learning, Reinforcement Learning, and Autonomous Learning. These layers allow AI to apply knowledge across contexts, learn from interactions, and independently gather and learn from information.

So, the journey from Broad AI to GenAI represents significant leaps in AI capabilities, moving towards AI systems that can truly understand, learn, and adapt like a human brain.

Let's explore a couple of AWS services that, from my perspective, are among the more popular today.

Amazon SageMaker:

Amazon SageMaker is a comprehensive platform that simplifies the machine learning workflow. It covers everything from data labeling and preparation to model training and deployment. Take advantage of SageMaker's Jupyter notebook integration for interactive data exploration and model development. The platform also supports popular ML frameworks like TensorFlow and PyTorch.

Amazon Q is a groundbreaking Generative AI assistant crafted with a focus on security and privacy. Its purpose is to unleash the transformative capabilities of this technology for employees within organizations of varying sizes and across diverse industries.

Introduces robust enhancements to the generative AI service, Amazon Bedrock.

Amazon Bedrock, an entirely managed service on AWS, provides access to extensive language models and other foundational models (FMs) from prominent artificial intelligence (AI) companies such as AI21, Anthropic, Cohere, Meta, and Stability AI, all consolidated through a unified API.

I would also like to share more information about Amazon Bedrock here about the innovations that were announced at the latest AWS re:Invent.

Fine-tuning for Amazon Bedrock:
Now, there are increased opportunities for model customization in Amazon Bedrock, featuring fine-tuning support for Cohere Command Lite, Meta Llama 2, and Amazon Titan Text models, with Anthropic Claude's support expected soon.

These recent enhancements to Amazon Bedrock significantly reshape how organizations, regardless of their size or industry, can leverage generative AI to drive innovation and redefine customer experiences.

AWS is compatible with all the leading deep-learning frameworks, facilitating their deployment. The deep-learning Amazon Machine Image, accessible on both Amazon Linux and Ubuntu, allows for the creation of managed, auto-scalable GPU clusters. This enables training and inference processes to be conducted at any scale. Also, AWS offers a range of AI services that allow you to integrate pre-trained models into your applications without the need for deep expertise in machine learning. Services like Amazon Rekognition for image and video analysis, Amazon Comprehend for natural language processing, and Amazon Polly for text-to-speech can enhance your applications with AI capabilities.

The best way to solidify your understanding of ML, AI, and GenAI on AWS is through hands-on projects. Start with simple projects and gradually increase complexity as you gain confidence. Use datasets available on platforms like Kaggle or create your own to train and test models.

Conclusion:

Embarking on a journey into Machine Learning, Artificial Intelligence, and Generative Artificial Intelligence on AWS is an exciting endeavor. By following these steps, you can lay a solid foundation for your understanding and proficiency in leveraging AWS services for ML and AI applications. Remember, the key to success is a combination of hands-on experience, continuous learning, and active engagement with the AWS community. Happy training!

Domain-Driven Design (DDD) in AWS. Find Your Business Domains.

Timur Galeev — Wed, 29 Mar 2023 15:27:12 +0000

This article is an introduction to Domain-Driven Design and how it can be used with AWS. I will provide guidance on how to define business domains in legacy monolithic applications and decompose them into a set of microservices step by step. By starting with Domain-Driven Design for your microservices, you can get the benefits of cloud scaling in your new refactored application.

Is Domain-Driven Design usefull for me?
The purpose of Domain-Driven Design is to free the domain code from technical details to have more room to work with its complexity. It is well suited to work with very complex domains and projects that are starting to dive into legacy.

Domain-Driven Design requires an understanding of the business idea or understanding of the final 'business product'. It requires time and commitment from both business experts and technical implementers. Domain-Driven Design should not be used in situations where you need 'quick solutions'. Instead, use Domain-Driven Design for software that supports the core business area rather than supporting areas. Running Domain-Driven Design can be achieved through an event-storming session. However, as mentioned above, this is a commitment worth making. It will allow you to develop software that is more tailored to the needs of your end clients. It will also help create decoupled services that are more scalable and maintainable. The combination will result in greater business agility.

Event Storming

Event Storming helps teams of business and technical people come to a consensus on what the solution should be. This happens without being distracted by the specific implementation details of how it will be implemented. This means, that it may take longer for the teams to start providing source code. However, all teams will be better aligned as to what each microservice should be responsible for. The event storming workshop is a brainstorming session. In this session all stakeholders in the solution work together to define the business events that correspond to the domains.
Suppose, we have a commerce development task where the business event might be a customer who applies for a new product. During the workshop, the group will begin to identify the object that triggered the event, the processes that should occur as a result, and any subsequent event triggered by the original event.

To do this, a team brainstorming session takes place where the event groups can identify areas of their business and then the contexts in which they operate. These can be used to define the usage and the relationship that occurs between each microservice and it’s context. Once the domains have been defined with the help of the business experts, the technical implementers can start designing the solution.

The result of the event-storming session is a domain model for development. The domain model can be used to define a number of bounded contexts.

Bounded Contexts

A bounded context is the boundary where each domain applies. The order contract opening example can be thought of as the 'Order Contract Opening Context' in a shop. In a complete system, there may be other contexts such as the product context, the description context and the manufacturing context. Identifying the business events that cause interactions between the different constrained contexts helps to determine how your microservices will interact with each other in the new architecture.

The example context map is just a sample of the core domains. There are also a number of supporting and subsequent domains. Although it is necessary to have a service that manages these, this is not part of the core application domain for sending products.

Core Domains and refactoring

When you start defining "Core Domains and Subsequent Domains" the question that usually arises is how to manage requests between domains.To do this, we'll look at options for using AWS services for how domains can be implemented.
Containerisation or serverless diagram of such solutions would rather be a 'modern architecture' than a good old-fashioned network and virtual machine deployment diagram. The advantage of these solutions is that the diagram itself helps to actually outline, what the logical functionality is, since we can be more expressive and fine-grained with resource usage.
The undisputed king of serverless computing platforms has been AWS LambdaAWS Lambda for several years now. It satisfies all the aforementioned conditions and can be used in a number of languages/implementations, including TypeScript.
Other viable options might include some of the more well-known container services such as AWS ECS or AWS EKS wrapped Fargate. However, they require considerably more setup and configuration, and also require that containerization actually takes place. It doesn't mean that containerisation is bad, in general containerisation can be good, it all depends on your development idea whether it's refactoring into microservices or starting a new application.
If you need Eventing then here it is Simple Notification Service (SNS). It is a push-based service, i.e. it automatically handles the distribution of the event to the recipients. SNS uses a pay-per-use model and it is essentially serverless as the only infrastructure you need is the SNS subject.
The modern cloud is about using its own API products to expose its applications, rather than building something of its own with Fastify, Kong or the like.
The API gateway acts as the only public interface connected to any other infrastructure, in our case primarily our Lambda compute functions, which will respond to paths defined in the gateway. In the case of AWS the service of interest, unsurprisingly, is called the AWS API Gateway.

The following diagram shows how monolith receives some of the traffic during the gradual addition of new microservices in the example application.

There is also AWS Migration Hub, it will help you in finding your domains and even offer AWS services that you can implement. This will help you to plan a refactoring or plan for migrating from your old OnPrem solutions to AWS with all modern solutions.

Conclusion

To summarise, people just don't tend to talk about 'domains' all day. Most employees do not pay attention to the implementation of domains in the organisation.
It is also worth noting that dividing systems into domains after they have been fully designed is also useless. DDD should be done, at least approximately, at the initial design stage.
But in any case, DDD is the place to be. In the example of using DDD in AWS it looks simple but when you start to go deeper here you find a lot of services where they all have to be interconnected and here comes the methods and dependencies. That's why it's very important to create a structure at the beginning of the work.

AWS, Terraform ,WordPress. Step-by-Step Guide Example

Timur Galeev — Tue, 24 Jan 2023 12:52:15 +0000

I've looked at many ways to run Wordpress on AWS but they are all expensive or unstable for me. So I decided to make a stable version of Wordpress installation that suited me.

Infrastructure management has changed a lot over the years. So much so that the traditional system administrator managed a rack full of servers. In many cases, the initial setup required manual intervention at the console.

Why AWS for WordPress might be a great choice
To begin with, AWS is a big deal. It's the cloud hosting provider with the largest market share. AWS is such a successful platform that it accounts for half of Amazon's operating income, which is in the billions of dollars.

AWS offers high scalability, making it ideal for sites with thousands of daily visitors. The platform also allows for any server configuration. This is ideal for high-performance sites such as online stores.

In this article, I will show you how to use Terraform to install Wordpress on AWS.

Our objectives

Create a ‘tf’ files which will hold all of our relevant configuration information (main.tf, …)
Define which provider we will be using in the Terraform config. (aws, cloudflare)
Certificate handling (AWS Certificate Manager)
Define security group rules and names.
Define the EC2 instances we want to create. (AWS Auto Scaling Group)
Run Terraform to plan and apply our configuration.

Before you get started, you’ll need to sign up for AWS. During the process, you’ll need to verify your account using a credit card – onto which they’ll charge $1 – and receive a verification code via SMS.
When you’re ready, select the Free support plan and you’ll get access to your console, which is where the magic happens

So, let's start creating our wordpress in AWS

I will assume that you have already configured all the necessary tools to run such as Terraform, aws-cli

The first thing you need to do is clone the repository:
$ git clone https://github.com/timurgaleev/wordpress-ec2-rds-alb-vpc.git

Once you've taken the templates from the repository, you now need to configure the 'tf' files, which will hold all of the relevant Terraform configuration. You can encapsulate it in one file, but for simplicity and convenience we will work within multiple files.

The structure of the repository is as follows:

acm.tf - AWS Certificate Manager Terraform module
alb.tf - AWS Application and Network Load Balancer Terraform module
asg.tf - AWS Auto Scaling Group (ASG) Terraform module
cloudflare.tf - Cloudflare Provider
efs.tf - Provides an Elastic File System (EFS) File System resource
output.tf - Terraform Output Values
rds.tf - AWS RDS Terraform module
security_group.tf - AWS EC2-VPC Security Group Terraform module
vpc.tf - AWS VPC Terraform module
variables.tf - variables used in Terraform.

Prerequisites needed for creating a envirenment
provider.tf you will need a change.

backend "s3" {
    bucket         = "ecs-terraform-examplecom-state"
    key            = "example/com.tfstate"
    region         = "eu-west-1"
    encrypt        = "true"
    dynamodb_table = "ecs-terraform-remote-state-dynamodb"
  }

It will also use an S3 bucket that will be used as a remote store for our Terraform state. This allows multiple users to work with one set of Infrastructure as Code without causing conflicts.

Then change the variables according to your needs.
variables.tf like environment, Domain, Cloudflare API Token

variable "asg_min_size" {
  description = "AutoScaling Group Min Size "
  default     = 1
}

variable "asg_max_size" {
  description = "AutoScaling Group Max Size "
  default     = 2
}

variable "asg_desired_capacity" {
  description = "AutoScaling Group Desired Capacity"
  default     = 1
}

variable "rds_engine" {
  description = "RDS engine"
  default     = "mariadb"
}

variable "rds_engine_version" {
  description = "RDS engine version"
  default     = "10.6.7"
}

variable "rds_instance_class" {
  description = "RDS instance class"
  default     = "db.t3.micro"
}

variable "site_domain" {
  description = "Domain"
  default     = "example.com"
}

variable "cloudflare_zone" {
  description = "cloudflare Zone Id"
}

variable "dns_ttl" {
  description = "cloudflare for dns = 1 is automatic."
  default     = 1
}

variable "dns_allow_overwrite_records" {
  description = "cloudflare allow overwrite records."
  default     = true
}

variable "cloudflare_api_token" {
  description = "cloudflare api token"
}

variable "ssh_key_name" {
  description = "SSH Key"
}

Now we can run our WordPress.

Run terraform init
Run terraform plan and review
Run terraform apply

Now enter your server domain, and you'll see the following WordPress installation screen. If that's the case, great job! If you don't see that screen, you may need to double-check the steps you followed.

You can destroy this WordPress by running:

terraform plan -destroy
terraform destroy  --force

Conclusion
We've only had a superficial look at how Terraform can be used in AWS, but I think a simple introduction is the best part! We created a database in RDS and plugged it into Wordpress as well as created and wired the necessary data into Cloudflare.