DEV Community: Pin Xiong

How Engineers Should Learn a New Technology in the AI Era

Pin Xiong — Tue, 13 Oct 2026 03:40:17 +0000

Originally published on xiongpin.dev.

Building a small website today takes about ten minutes. A landing page, a form, a data table to store the submissions and an admin view to read them. You describe what you want, an AI coding tool writes the code, and then it deploys the service.

Twenty years ago the same website started with a harder question: where does the code run? You picked a machine, installed an operating system, configured the runtime, opened the network ports and set up a way to release changes.

New technology keeps arriving faster

That question has been answered six times in thirty years.

Two things stand out in that timeline.

First, no generation kills the one before it. Each one builds on the last and solves what it left behind, so putting software into production keeps getting easier.

Second, the gaps turn from years into months. The first stages sit roughly ten, five and seven years apart; the last three, from Docker in March 2013 to Lambda in November 2014, all land inside twenty months.

Then the chain collapsed into a single step. An AI coding tool can do the whole thing now: it writes the code, sets up the environment and deploys the service. The same website that once needed all of that preparation comes together in minutes. The point is the speed, not the tool.

How to understand a technology

Before we talk about how to learn a new technology, we should look at how to understand one. Deployment is not a single technology, and neither are Kubernetes and AI coding. We talk about each as if the name covered one skill, but what sits under it is not the same kind of thing: some are notations you memorize, some are products you choose, some are philosophies you reason about, and some are mechanisms. Some turn over every few months; others have not changed in decades. Modeling a technology in layers is what makes that visible.

The four layers

The four layers are a way to take a technology apart. Splitting it shows where each piece lands: a single capability tends to sit in one layer, while a product reaches across several. That is what makes the real questions answerable: how to learn it, whether it is worth learning and how much time it deserves.

How to learn each layer

Take the first of those questions, how to learn it. The strategy is different for every layer, but each one comes down to the same questions: which direction to work in, how deep to go and how you know you have gone far enough.

That is the whole model: four layers, each with its own way in, its own depth and its own test. It holds for any technology that is new to you.

How to understand AI coding

The model stays abstract until it lands on a technology you actually use. AI coding is the obvious place to test the model, because almost everyone is already working with it. When a new technology arrives, there is one judgment to make: split it, see which layers it spans and learn each layer in its own way. You do not have to label yourself top-down or bottom-up first. In the AI era the two have largely merged, because AI hands you the surface in seconds and explains the mechanism on demand, so there is little to gain by picking a side in advance.

Where AI coding lands

At the framework layer: Cursor, Claude Code, Kiro and Qoder.

The picture says one thing clearly: AI coding spans all four layers, so "should I learn AI coding" is the wrong question. It breaks into four.

How to learn it, layer by layer

How to learn in the AI era

Three points to end on. The first two sit at opposite ends of the same model, and they answer the other two questions: whether it is worth learning, and how much time it deserves.

Bet early on the top two layers

Usage and tooling have always been the fastest to turn over. Getting a bet wrong there now costs days instead of months, so there is no reason to wait for a technology to settle. Place the bet early and place several at once. All of that starts with an open mind toward whatever is new.

Hold on to the bottom two

Paradigm and mechanism barely move. Knowledge from the top layers expires with the tools; the principles underneath do not. That payoff takes the longest to arrive, which is why you keep the understanding rather than a memory of details.

Abstract whatever you learn

Understanding how a technology works is half of it. The other half is pulling the principle out into a shape that is not tied to that one technology, so it carries into the next one.

That is the answer: not the newest tool, and not the fundamentals alone, but both, plus the abstraction that lets you carry what you learn forward.

AI Coding in Practice: Refactoring a Zero-Docs Microservice

Pin Xiong — Thu, 08 Oct 2026 04:19:31 +0000

Originally published on xiongpin.dev.

Most backend teams have a service that runs in production and plays a pivotal role in the business. However, nobody knows the full picture of how it behaves. We also have a service like this. It is a digital-key microservice acting as a piece of infrastructure that lets the mobile device lock and unlock a car. It has already run for two years, but has zero documentation. Until recently, the only complete description of how this service behaved was carried in the heads of a few engineers, some of whom moved to other projects.

This article is a field report showing how we used AI coding to refactor this service. The idea of how to refactor it comes from Kiro's design philosophy. We designed 13 steps to make the refactoring process successful. You can benefit from what we learned: what we did well, what we didn't and the results we achieved. The short version: coding time dropped from one week to 4 hours, work we scoped for two people over a month was finished by one person in two weeks, and 90% of the functions passed their first test. One thing we want to emphasize is that it's not a demo, but a real service running in production with zero tolerance for incidents.

The problem: a service nobody dared to touch

This service spans three ends: the mobile device (Apple, Xiaomi and Huawei), the vehicle (TSP, module and RKE) and the cloud (API, PKI and certificates). Its three communication paths are:

The mobile device connects to the vehicle over BLE, NFC and UWB.
The mobile device connects to the cloud over HTTPS.
The vehicle connects to the cloud over MQTT.

Every unlock, every permission grant and every key lifecycle event goes through these three ends.

Three properties made it dangerous to touch:

It's production-critical. A failed key exchange means a customer is left standing next to a car that won't open, under a zero-tolerance SLA.
It has zero documentation. There are no records of two years of change requests, urgent bugfixes and performance optimization. The only documentation is the code itself and the memory that lives in engineers' heads.
Knowledge is leaving. The core engineers who programmed it have moved to other projects. Every question about "why this code is this way" had a half-day turnaround.

The cost showed up in the worst place: debugging. Tracing a cross-end problem (an inconsistent status issue that can only be reproduced when the mobile device, the vehicle and the cloud interact in a specific order) took more than two weeks. Not because the fix was hard, but because understanding the system was hard: there was no call-chain map, no flowcharts, no database schema design docs.

At the same time, business changes are queued up for this service, including internationalization, a batch of new product features and a migration of the service onto our company foundation framework. In practice, that migration consists of replacing Redis with Redisson and externalizing runtime configuration to Nacos. Every requirement means several weeks of learning the code logic before we can change anything. The only solution is spec-driven AI coding — letting AI understand the service first, splitting a requirement into several small specs and then writing code.

Why AI coding — and why not just let it loose

Why did we choose AI coding here? The decision came down to four observations:

The code is the only source of truth — and AI can read it fast. The code repository is the only trustworthy description of the system, with zero documentation. A person can read only a few hundred lines of code an hour. However, AI can read the whole code repository in minutes. "Zero documentation" is a fatal issue for a human but not for AI — the precondition is that AI's understanding of the source code must be confirmed by a human.
Most of the work is mechanical, not creative. Things like "migrating the service to the company foundation framework", "documenting the call chain" and "generating test cases" are repetitive and well-defined tasks once we have a precise description of the existing behavior.
The bottleneck is comprehension, not typing speed. AI writes code far faster than the team can keep up. The real question, though, was not the model. It was a disciplined workflow: turning "understanding the code, then writing code to meet business requirements" into artifacts a human can review.
AI tool cost is covered by the company. We can use the internal AI coding environment, including its tokens and support. The only real investment was process design.

In other words, the hardest part was never which LLM to pick, but the design of the full workflow: making a probabilistic code generator trusted in a zero-tolerance system.

In the end, we surveyed AI coding products on the market and found that Kiro's design philosophy came closest to meeting our needs. We read Kiro's official docs closely before refactoring, and then we designed the solution around our own service, not around the tool's defaults.

Kiro's design philosophy

Kiro's design philosophy is built on three pillars: distilling proven engineering patterns, respecting developer conventions and maximizing AI capabilities. Those pillars show up as four core capabilities:

Specs: The Specs feature defines three stages crossing requirements (requirements.md), design (design.md) and tasks (tasks.md), with human approval between them. "Prompt" is your wish, but "Spec" is your contract.
Steering: The project's basic rules are written once and are loaded into every interaction, including the product's knowledge, technical specifications and the project's structure. In contrast to AGENTS.md, Kiro supports inclusion modes where agents can run according to a specific mode including always, fileMatch, manual and auto.
Hooks: Hooks let you define an event mechanism to build a workflow automatically. All behaviors that happen in a session are defined as different event types that can trigger actions you define in advance.
Custom Agents: Custom agents are open for developers to design the agent's behavior to meet your needs. What you can set up for your agent includes pre-approving specific tools, limiting tool access, providing relevant context and configuring tool behavior.

In short, three ideas from those capabilities shaped our process: the quality of your context sets the ceiling of AI output; restrictions limit what AI is allowed to do; and different agents serve different purposes.

The process: 13 steps across four phases

The full flow consists of 13 steps divided into 4 phases. In the diagram below, steps marked with an asterisk are each checked by a separate agent, and every step ends with human review.

Phase 1 — Understand (Steps 1–3)

Step 1: Generate tech docs. AI read the whole codebase and produced architectural docs, per-flow documents and module maps. This was the moment that tribal memory was written down: the system had its own records that didn't depend on anyone.

Step 2: Generate test cases. Based on the docs from Step 1, AI generated test cases for APIs and the coverage reached 98%. Why did this step come before any refactoring? Because tests must pin down the current and verified behavior, not AI's intention. As a result, every later refactor had a fixed baseline to be verified against, not an opinion.

Step 3: Deep code distillation. AI sorted out the call chain per business scenario, the distribution of error codes, core data structures and the Redis key inventory. This step can turn a two-week investigation into a four-hour one.

Phase 2 — Define (Step 4)

Step 4: Code standards + harness engineering. We set up the coding standards, module boundaries, the documentation structure (specs, plans, architecture and guides) and the AI coding workflow itself before AI wrote code. This step limited AI's behavior by telling it what, when and how to do it.

Phase 3 — Refactor & verify (Steps 5–10)

Step 5: AI refactor. AI wrote code based on the foundation framework and executed against the specs and standards. The agent never decided when it was done; the Phase 2 contract did.

Steps 6–8: Three independent verifications. A consistency-check agent compared the new code against the distilled docs ("does the implementation still match the documented behavior?"). A regression agent ran the Step-2 test suite against the new code. A code-diff agent reviewed the full diff for anything that changed unintentionally. Each check ran in its own agent with fresh context. All three passed through human review before the next phase opened.

Steps 9–10: Write it back down. The changes, the optimizations and the known TODOs were documented as part of the work. Everything flowed into the knowledge base. Which is why the refactor is durable: the next AI session — or the next engineer — starts from living docs, not from zero.

The commit history tells the same story from the other side: design documents first, implementation second.

Phase 4 — Deploy (Steps 11–13)

Step 11: Deploy & integration. In contrast to a typical migration, the first full run in the new environment took 20 minutes — the framework migration was already validated by the Phase 3 gates, so integration was confirmation, not discovery.

Step 12: Grayscale plan. The rollout plan was written as a spec and approved by a human gate before execution: which traffic slices, in which order, with which abort criteria.

Step 13: Grayscale release. We executed the plan. Zero production incidents.

What worked well

10x coding productivity increase. The rewrite cut coding time per task from one week to 4 hours, because the distilled documents and the Step 2 test suite gave AI a clear contract to code against. The saved hours did not disappear; they moved to review, which is where the bottleneck now sits.
4x fewer person-months. We scoped the migration for 2 people over 1 month; 1 person finished it in 2 weeks. The gain came from the documentation phase, not from faster typing.
90% first-pass rate. The Step 2 suite covered 98% of the API surface; 90% of the functions then passed the first 20-minute test. Only two bugs were found throughout the entire process; both were caused by humans. That told us where to invest next: better specs, not a better model.

How we measured

These numbers come from one service, one team and one release, so read them as a field report, not as a controlled experiment.

The comparison is task to task, not run to run. Before means the same class of change done the old way: reading the code and asking the engineers who wrote it. After means the same change done through the 13-step workflow. The two figures were tracked by the team, by hand. No timers were involved.
Coding time excludes review. Review is not inside the 4 hours, and it is now our bottleneck. The end-to-end saving is real, but smaller than 10x.
The test suite was generated from the Step 1 docs. A behavior the docs describe wrongly is therefore a behavior the suite cannot catch. That is why the critical-path specs were approved by humans against business intent, not only distilled from the code.
Tokens were free. We ran on an internal AI coding environment. Token cost is covered centrally by the company, so it is not part of these results.

What didn't work

Small changes need a full cycle. Even a tiny change requires running a full workflow.
Authorization interruptions. Executing a long task needs multiple authorization confirmations, which interrupt the work rhythm.
Review fatigue. AI produced output far faster than we could review it.
Player vs. referee conflict. The same agent wrote the code and reviewed its own work, so quality degraded. As a result, production and verification must be executed by two separate agents.
Doc-dependent bug analysis. AI analyzed the bug according to the documentation first instead of the code.

These pain points are process issues, not AI capability limits. Better workflows solve most of them.

Key takeaways

Foundational principles

Code without docs is tech debt. Documentation is the requirement, the unfinished task. Code is merely the result. As a result, the debt stays invisible until something breaks.
Process > AI code generation. The workflow and standards you design matter more than letting AI write code.
Best model for docs, cheaper for code. Use the strongest model for documentation generation. In contrast, a standard model is sufficient for code writing.
Review docs > review code. Reviewing documentation catches more issues than reviewing code. Documentation exposes hidden bugs, because the code is written from the docs.

Practical wisdom & mindset

Don't rush to manual analysis. When bugs appear, let AI analyze first: output full call chains as docs, read the docs rather than the code, then let AI fix. This works as long as the docs are sound; the doc caveat from the previous section still applies.
Your taste sets the ceiling. AI generates code at the level of your code taste. Better code taste means higher AI output quality.
Attitude determines outcomes. Positive people find ways to help AI succeed. Neutral or negative people spend time finding faults.
Unreasonable becomes reasonable. After refactoring, previously unreasonable product demands suddenly become feasible. Better harness engineering unlocks new possibilities.

The model is a component; the process is the product.

Easy to use ECS

Pin Xiong — Fri, 15 Jul 2022 14:00:09 +0000

Elastic Container Service(ECS) is a highly scalable and fast container management service that makes it easy to run, stop, and manage containers on a cluster.

Today, I'll show you how to use it!

Architecture

There are several steps as below, and all of source codes have been put in my github repo(infrastructure and golang-web).

Prerequisites

Create an aws account with the IAM AdministratorAccess permission
Install and configure aws CLI
Install and configure Terraform

Setup environment

Terraform saves all changes in *.tfstate file, so we'd better store *.tfstate in aws s3 bucket instead of local machine. This step will build a aws s3 bucket to store *.tfstate file.

Init Terraform

Download infrastructure first

$ cd setup
$ terraform init

Create s3 bucket

$ terraform apply

You will be prompted to enter an aws region code, such as us-east-1. After that, you need to make sure the listed resources that will be crated and then enter yes

You can see the output s3_bucket_terraform_state below

Outputs:

s3_bucket_terraform_state = "**********-us-east-1"

Build resources

Now, we begin to build the resources including VPC, subnets, ECS and Pipeline.

Setup remote backup

$ cd ../region/virginia

and then, update the block of s3 in providers.tf file

backend "s3" {
    bucket  = "**********-us-east-1"
    key     = "terraform/backend.tfstate"
    region  = "us-east-1"
    encrypt = "true"
  }

The following things maybe need to be modified:

Set bucket as the output of s3_bucket_terraform_state
Set key as the path to store *.tfstate file in s3 bucket
Update region as the region code that you entered when creating s3 bucket above
Set encrypt as true

Create resources

You can modify the configuration in the main.tf file according to your needs, and then run the following commands

$ terraform init
$ terraform apply

You will be prompted to enter yes after confirmed the listed resources.

After 10 minutes, all resources are created.

Deploy your code

Download golang-web first and then set your git remote as below

$ git remote add aws ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/golang-web

$ git remote -v

$ git push aws master:master
Enumerating objects: 62, done.
Counting objects: 100% (62/62), done.
Delta compression using up to 12 threads
Compressing objects: 100% (55/55), done.
Writing objects: 100% (62/62), 25.36 KiB | 6.34 MiB/s, done.
Total 62 (delta 26), reused 3 (delta 0), pack-reused 0
To ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/golang-web
 * [new branch]      master -> master

Now, everything is done, let's take a look at the outputs in aws.

Outputs

First, we can see VPC and subnet have been created.

VPC
Subnets

and then the codepipe has been built

CodeCommit
CodeBuild
CodePipeline
ECR

we also can see that a public endpoint is created.

ELB
Target group

Now, It's time to see CES

ECS Dashboard
ECS Cluster
ECS Service
Task Definition

Verification

Access public endpoint
Scale out
Access start api will exhaust CPU resource serval times

and then see the monitor in Cloud watch

the ecs has been created 2 new tasks

Scale in Access stop api will release CPU resource serval times

and then see the monitor in Cloud watch

the ecs has been draining tasks

How to build EKS with Terraform

Pin Xiong — Sat, 23 Apr 2022 12:32:15 +0000

ESK(Amazon Elastic Kubernetes Service) is a fully managed Kubernetes cluster by aws. Now, we want to build EKS with Terraform. All the codes have been put in github.

Preparation

We need to build VPC first and then build EKS, you can refer to How to build VPC with Terraform

Code structure

infrastructure
├── module
│   ├── networking
│   ├── eks
│   │   ├── main.tf
│   │   ├── outputs.tf
│   │   └── variables.tf
│   └── jumpserver
│       ├── main.tf
│       ├── outputs.tf
│       └── variables.tf
├── region
│   └── virginia
│       ├── main.tf
│       └── providers.tf
└── setup
    ├── main.tf
    ├── outputs.tf
    ├── providers.tf
    └── variables.tf

EKS

Now, we begin to build EKS, the source codes in eks folder are listed as below:

main.tf

locals {
  name               = var.name
  vpc_id             = var.vpc_id
  public_subnets_id  = var.public_subnets_id
  private_subnets_id = distinct(flatten(var.private_subnets_id))
  desired_size       = var.desired_size
  max_size           = var.max_size
  min_size           = var.min_size
  security_group_ids = var.security_group_ids
  tags               = var.eks_tags
}

# Define the role to be attached EKS
resource "aws_iam_role" "eks_role" {
  name               = "eks-role"
  assume_role_policy = jsonencode({
    "Version" : "2012-10-17",
    "Statement" : [
      {
        "Effect" : "Allow",
        "Action" : [
          "sts:AssumeRole"
        ],
        "Principal" : {
          "Service" : [
            "eks.amazonaws.com"
          ]
        }
      }
    ]
  })
  tags = merge({
    Name : "EKS Role"
  }, local.tags)
}

# Attach the CloudWatchFullAccess policy to EKS role
resource "aws_iam_role_policy_attachment" "eks_CloudWatchFullAccess" {
  policy_arn = "arn:aws:iam::aws:policy/CloudWatchFullAccess"
  role       = aws_iam_role.eks_role.name
}

resource "aws_iam_role_policy_attachment" "eks_AmazonEKSClusterPolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.eks_role.name
}

# Optionally, enable Security Groups for Pods
resource "aws_iam_role_policy_attachment" "eks_AmazonEKSVPCResourceController" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSVPCResourceController"
  role       = aws_iam_role.eks_role.name
}

# Default Security Group of EKS
resource "aws_security_group" "security_group" {
  name        = "${local.name} Security Group"
  description = "Default SG to allow traffic from the EKS"
  vpc_id      = local.vpc_id

  ingress {
    from_port       = "0"
    to_port         = "0"
    protocol        = "TCP"
    security_groups = local.security_group_ids
  }

  tags = merge({
    Name = "${local.name} Security Group"
  }, local.tags)

}

# EKS Cluster
resource "aws_eks_cluster" "eks" {
  name    = local.name
  version = "1.22"

  enabled_cluster_log_types = [
    "api",
    "audit",
    "authenticator",
    "controllerManager",
    "scheduler"
  ]
  role_arn = aws_iam_role.eks_role.arn

  timeouts {}

  vpc_config {
    endpoint_private_access = true
    endpoint_public_access  = true
    public_access_cidrs     = [
      "0.0.0.0/0",
    ]
    security_group_ids = [
      aws_security_group.security_group.id
    ]
    subnet_ids = flatten([var.public_subnets_id, var.private_subnets_id])
  }

  tags = merge({
    Name = local.name
  }, local.tags)
}

resource "aws_iam_role" "node_group_role" {
  name                  = format("%s-node-group-role", lower(aws_eks_cluster.eks.name))
  path                  = "/"
  force_detach_policies = false
  max_session_duration  = 3600
  assume_role_policy    = jsonencode(
    {
      Statement = [
        {
          Action    = "sts:AssumeRole"
          Effect    = "Allow"
          Principal = {
            Service = "ec2.amazonaws.com"
          }
        },
      ]
      Version = "2012-10-17"
    }
  )
}

resource "aws_iam_role_policy_attachment" "node_group_AmazonEC2ContainerRegistryReadOnly" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
  role       = aws_iam_role.node_group_role.id
}

resource "aws_iam_role_policy_attachment" "node_group_AmazonEKSWorkerNodePolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"
  role       = aws_iam_role.node_group_role.id
}

resource "aws_iam_role_policy_attachment" "node_group_AmazonEC2RoleforSSM" {
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonEC2RoleforSSM"
  role       = aws_iam_role.node_group_role.id
}

resource "aws_iam_role_policy_attachment" "node_group_AmazonEKS_CNI_Policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
  role       = aws_iam_role.node_group_role.id
}

resource "aws_iam_role_policy_attachment" "node_group_CloudWatchAgentServerPolicy" {
  policy_arn = "arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy"
  role       = aws_iam_role.node_group_role.id
}

resource "aws_eks_node_group" "node_group" {
  cluster_name  = aws_eks_cluster.eks.name
  disk_size     = 0
  capacity_type = "SPOT"
  labels        = {
    "eks/cluster-name"   = aws_eks_cluster.eks.name
    "eks/nodegroup-name" = format("nodegroup_%s", lower(aws_eks_cluster.eks.name))
  }
  node_group_name = format("nodegroup_%s", lower(aws_eks_cluster.eks.name))
  node_role_arn   = aws_iam_role.node_group_role.arn

  subnet_ids = local.private_subnets_id

  scaling_config {
    desired_size = local.desired_size
    max_size     = local.max_size
    min_size     = local.min_size
  }

  timeouts {}

  lifecycle {
    create_before_destroy = true
  }

  tags = merge({
    Name                 = local.name
    "eks/cluster-name"   = local.name
    "eks/nodegroup-name" = format("%s Node Group", aws_eks_cluster.eks.name)
    "eks/nodegroup-type" = "managed"
  }, local.tags)
}

variables.tf

variable "name" {
  description = "The EKS name"
}

variable "vpc_id" {
  type        = string
  description = "The VPC id"
}

variable "public_subnets_id" {
  description = "The public subnets id"
  type        = list
}

variable "private_subnets_id" {
  description = "The private subnets id"
  type        = list
}

variable "desired_size" {
  description = "The desired size of node"
  type        = number
  default     = 1
}

variable "max_size" {
  description = "The maximum size of node"
  type        = number
}

variable "min_size" {
  description = "The minimum size of node"
  type        = number
  default     = 0
}

variable "security_group_ids" {
  description = "The security groups to access EKS"
  type        = list
  default     = []
}

variable "eks_tags" {
  description = "A map of tags to add to EKS"
  type        = map(string)
  default     = {}
}

outputs.tf

output "endpoint" {
  value = aws_eks_cluster.eks.endpoint
}

Jump Server

We cannot access EKS directly, so we can use a EC2 as jump server to do it. The source codes in jumpserver folder are listed as below:

main.tf

locals {
  vpc_id             = var.vpc_id
  instance_type      = var.instance_type
  instance_ami       = var.instance_ami
  subnet_id          = var.public_subnets_id[0][0]
  security_group_ids = var.security_group_ids
  tags               = var.shared_tags
}

# Define the security group for jump server.
resource "aws_security_group" "jump-server" {
  name   = "Security group for jump server"
  vpc_id = local.vpc_id

  ingress {
    from_port   = 22
    protocol    = "TCP"
    to_port     = 22
    cidr_blocks = ["0.0.0.0/0"]
    description = "Allow SSH inbound traffic"
  }

  # Allow all outbound traffic
  egress {
    from_port        = 0
    protocol         = "-1"
    to_port          = 0
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
    description      = "Allow all outbound traffic"
  }

  tags = merge({
    Name : "Security group for jump server"
  }, local.tags)
}

# Define the role to be attached ec2 instance of the jump server
resource "aws_iam_role" "jump-server" {
  name               = "jump-server-role"
  assume_role_policy = jsonencode({
    "Version" : "2012-10-17",
    "Statement" : [
      {
        "Effect" : "Allow",
        "Action" : [
          "sts:AssumeRole"
        ],
        "Principal" : {
          "Service" : [
            "ec2.amazonaws.com"
          ]
        }
      }
    ]
  })
  tags = merge({
    Name : "Jump Server Role"
  }, local.tags)
}

# Attach the AdministratorAccess policy to jump server role
resource "aws_iam_role_policy_attachment" "jump-server-AdministratorAccess" {
  policy_arn = "arn:aws:iam::aws:policy/AdministratorAccess"
  role       = aws_iam_role.jump-server.name
}

# Define iam instance profile for ec2 instance of the message transmission service
resource "aws_iam_instance_profile" "jump-server-profile" {
  name = "jump-server-profile"
  role = aws_iam_role.jump-server.name
  tags = merge({
    Name : "Jump Server Profile"
  }, local.tags)
}

# Define the ec2 instance
resource "aws_instance" "jump-server" {
  count                       = 1
  instance_type               = local.instance_type
  ami                         = local.instance_ami
  associate_public_ip_address = true
  hibernation                 = false
  subnet_id                   = local.subnet_id
  vpc_security_group_ids      = setunion([aws_security_group.jump-server.id], local.security_group_ids)
  iam_instance_profile        = aws_iam_instance_profile.jump-server-profile.name
  tags                        = merge({
    Name = "Jumper Server"
  }, local.tags)
}

variables.tf

variable "vpc_id" {
  type = string
}

variable "instance_type" {
  type    = string
  default = "t3.micro"
}

variable "instance_ami" {
  type    = string
  default = "ami-0f9fc25dd2506cf6d"
}

variable "public_subnets_id" {
  description = "The public subnet id"
  type        = list
}

variable "security_group_ids" {
  description = "The security group for jump server"
  type        = list
  default = []
}

variable "shared_tags" {
  description = "A map of tags to add to Jump Server"
  type        = map(string)
  default     = {}
}

outputs.tf

output "jump-server-security-group-id" {
  value = aws_security_group.jump-server.id
}

Virginia

The source code region/virginia folder are listed as below:

providers.tf

terraform {
  required_version = ">= 0.13.7"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.8.0"
    }
    random = {
      source  = "hashicorp/random"
      version = "3.1.0"
    }
    null = {
      source  = "hashicorp/null"
      version = "3.1.0"
    }
    local = {
      source = "hashicorp/local"
      version = "2.1.0"
    }
    template = {
      source = "hashicorp/template"
      version = "2.2.0"
    }
    # Kubernetes Provider
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "2.1.0"
    }
  }

  backend "s3" {
    bucket  = "pin-terraform-state-us-east-1"
    key     = "terraform/backend.tfstate"
    region  = "us-east-1"
    encrypt = "true"
  }
}

main.tf

locals {
  region             = "us-east-1"
  availability_zones = ["${local.region}a", "${local.region}b", "${local.region}c"]
  tags               = {
    "Environment" : "PROD"
    "Project" : "Infrastructure"
  }
}

provider "aws" {
  region = local.region
}

module "Networking" {
  source                     = "../../module/networking"
  name                       = "VPC"
  availability_zones         = local.availability_zones
  vpc_cidr_block             = "10.0.0.0/16"
  public_subnets_cidr_block  = ["10.0.32.0/24", "10.0.96.0/24", "10.0.224.0/24"]
  private_subnets_cidr_block = ["10.0.0.0/19", "10.0.64.0/19", "10.0.128.0/19"]
  vpc_tags                   = local.tags
}

module "JumpServer" {
  source            = "../../module/jumpserver"
  vpc_id            = module.Networking.vpc_id
  public_subnets_id = module.Networking.public_subnets_id
  shared_tags       = local.tags
}

module "EKS" {
  source             = "../../module/eks"
  name               = "EKS"
  vpc_id             = module.Networking.vpc_id
  public_subnets_id  = module.Networking.public_subnets_id
  private_subnets_id = module.Networking.private_subnets_id
  desired_size       = 4
  max_size           = 16
  eks_tags           = local.tags
}

Now, we can build it

$ terraform init

$ terraform apply

Result

How to build VPC with Terraform

Pin Xiong — Wed, 20 Apr 2022 14:39:31 +0000

It's one of the most important things that everyone need to build VPC after applied a new aws account. The purpose of this blog is to help you build it using Terraform. All the codes have been put in github.

Design for networking

Public subnet: Deploying websites can be accessed directly by the Internet
Private subnet: Deploying applications only can be accessed by the websites in Public subnet. Also, the application can access to the Internet through NAT Gateway

How to build

Before creating a VPC, we need to install TerraForm first.

Code structure

infrastructure
├── module
│   └── networking
│       ├── main.tf
│       ├── outputs.tf
│       └── variables.tf
├── region
│   └── virginia
│       ├── main.tf
│       └── providers.tf
└── setup
    ├── main.tf
    ├── outputs.tf
    ├── providers.tf
    └── variables.tf

Setup environment

The source codes in setup folder are listed as below:

main.tf

locals {
  region = var.region
}

# Define a s3 bucket to store terraform state file.
resource "aws_s3_bucket" "terraform_state" {
  bucket        = format("terraform-state-%s", local.region)
  force_destroy = false
  lifecycle {
    ignore_changes = [bucket]
  }
}

resource "aws_s3_bucket_versioning" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.bucket
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

outputs.tf

output "s3_bucket_terraform_state" {
  value = aws_s3_bucket.terraform_state.bucket
}

providers.tf

terraform {
  required_version = ">= 0.13.7"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.8.0"
    }
  }
}

variables.tf

variable "region" {
  description = "The name of the aws Region"
  type        = string
}

And then, we need to create s3 bucket for remote backend, the s3 bucket name is the output of s3_bucket_terraform_state.

$ terraform init

$ terraform apply

Networking

The source codes in networking folder are listed as below:

main.tf

locals {
  name                       = var.name
  cidr_block                 = var.vpc_cidr_block
  public_subnets_cidr_block  = var.public_subnets_cidr_block
  private_subnets_cidr_block = var.private_subnets_cidr_block
  availability_zones         = var.availability_zones
  tags                       = var.vpc_tags
}

# VPC
resource "aws_vpc" "vpc" {
  cidr_block           = local.cidr_block
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = merge( {
    Name = local.name
  }, local.tags)
}

# Subnets
# Internet Gateway for Public Subnet
resource "aws_internet_gateway" "internet_gateway" {
  vpc_id = aws_vpc.vpc.id
  tags   = merge({
    Name = "${local.name} Internet Gateway"
  }, local.tags)
}

# EIP for NAT
resource "aws_eip" "nat_eip" {
  vpc        = true
  depends_on = [aws_internet_gateway.internet_gateway]
}

# NAT
resource "aws_nat_gateway" "nat_gateway" {
  allocation_id = aws_eip.nat_eip.id
  subnet_id     = element(aws_subnet.public_subnet.*.id, 0)

  tags = merge({
    Name = "${local.name} Nat Gateway"
  }, local.tags)
}

# Public subnet
resource "aws_subnet" "public_subnet" {
  vpc_id                  = aws_vpc.vpc.id
  count                   = length(local.public_subnets_cidr_block)
  cidr_block              = element(local.public_subnets_cidr_block, count.index)
  availability_zone       = element(local.availability_zones, count.index)
  map_public_ip_on_launch = true
  tags                    = merge( {
    Name = "${local.name} Public Subnet ${element(var.availability_zones, count.index)}"
  }, local.tags)
}

# Private Subnet
resource "aws_subnet" "private_subnet" {
  vpc_id                  = aws_vpc.vpc.id
  count                   = length(local.private_subnets_cidr_block)
  cidr_block              = element(local.private_subnets_cidr_block, count.index)
  availability_zone       = element(local.availability_zones, count.index)
  map_public_ip_on_launch = false
  tags                    = merge({
    Name = "${local.name} Private Subnet ${element(var.availability_zones, count.index)}"
  }, local.tags)
}


# Routing tables to route traffic for Private Subnet
resource "aws_route_table" "private" {
  vpc_id = aws_vpc.vpc.id

  tags = merge({
    Name = "${local.name} Private Route Table"
  }, local.tags)
}

# Routing tables to route traffic for Public Subnet
resource "aws_route_table" "public" {
  vpc_id = aws_vpc.vpc.id

  tags = merge({
    Name = "${local.name} Public Route Table"
  }, local.tags)
}

# Route for Internet Gateway
resource "aws_route" "public_internet_gateway" {
  route_table_id         = aws_route_table.public.id
  destination_cidr_block = "0.0.0.0/0"
  gateway_id             = aws_internet_gateway.internet_gateway.id
}

# Route for NAT
resource "aws_route" "private_nat_gateway" {
  route_table_id         = aws_route_table.private.id
  destination_cidr_block = "0.0.0.0/0"
  nat_gateway_id         = aws_nat_gateway.nat_gateway.id
}

# Route table associations for Public Subnets
resource "aws_route_table_association" "public" {
  count          = length(local.public_subnets_cidr_block)
  subnet_id      = element(aws_subnet.public_subnet.*.id, count.index)
  route_table_id = aws_route_table.public.id
}

# Route table associations for Private Subnets
resource "aws_route_table_association" "private" {
  count          = length(local.private_subnets_cidr_block)
  subnet_id      = element(aws_subnet.private_subnet.*.id, count.index)
  route_table_id = aws_route_table.private.id
}

# Default Security Group of VPC
resource "aws_security_group" "security_group" {
  name        = "${local.name} Security Group"
  description = "Default SG to allow traffic from the VPC"
  vpc_id      = aws_vpc.vpc.id
  depends_on  = [
    aws_vpc.vpc
  ]

  ingress {
    from_port = "0"
    to_port   = "0"
    protocol  = "-1"
    self      = true
  }

  egress {
    from_port = "0"
    to_port   = "0"
    protocol  = "-1"
    self      = "true"
  }

  tags = merge({}, local.tags)
}

outputs.tf

output "vpc_id" {
  value = aws_vpc.vpc.id
}

output "public_subnets_id" {
  value = [aws_subnet.public_subnet.*.id]
}

output "private_subnets_id" {
  value = [aws_subnet.private_subnet.*.id]
}

output "security_groups_ids" {
  value = [aws_security_group.security_group.id]
}

output "public_route_table" {
  value = aws_route_table.public.id
}

variables.tf

variable "name" {
  description = "The VPC name"
}

variable "vpc_cidr_block" {
  description = "CIDR block of the vpc"
}

variable "public_subnets_cidr_block" {
  type        = list
  description = "CIDR block for Public Subnet"
}

variable "private_subnets_cidr_block" {
  type        = list
  description = "CIDR block for Private Subnet"
}

variable "availability_zones" {
  type        = list
  description = "AZ in which all the resources will be deployed"
}

variable "vpc_tags" {
  description = "A map of tags to add to VPC"
  type        = map(string)
  default     = {}
}

Build VPC

The source codes in virginia folder are listed as below:

main.tf

locals {
  // the region code of virginia
  region             = "us-east-1"
  availability_zones = ["${local.region}a", "${local.region}b", "${local.region}c"]
  tags               = {
    "Environment" : "PROD"
    "Project" : "Infrastructure"
  }
}

provider "aws" {
  region = local.region
}

module "Networking" {
  source                     = "../../module/networking"
  name                       = "VPC"
  availability_zones         = local.availability_zones
  vpc_cidr_block             = "10.0.0.0/16"
  public_subnets_cidr_block  = ["10.0.32.0/24", "10.0.96.0/24", "10.0.224.0/24"]
  private_subnets_cidr_block = ["10.0.0.0/19", "10.0.64.0/19", "10.0.128.0/19"]
  vpc_tags                   = local.tags
}

providers.tf

terraform {
  required_version = ">= 0.13.7"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.8.0"
    }
  }

  backend "s3" {
    // the s3 bucket name
    bucket  = "*********"
    key     = "terraform/backend.tfstate"
    region  = "us-east-1"
    encrypt = "true"
  }
}

Now, we can build the VPC

$ terraform init

$ terraform apply