Observabilidade de microsserviços com OpenTelemetry e Amazon OpenSearch [Lab Session]

OpenTelemetry is a collection of tools, APIs, and SDKs. Use it to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) to help you analyze your software’s performance and behavior.
https://opentelemetry.io/

Referências/Reference:
https://catalog.us-east-1.prod.workshops.aws/workshops/1abb648b-2ef8-442c-a731-efbcb69c1e1e/en-US
https://github.com/aws-samples/observability-with-amazon-opensearch.git
https://github.com/idealo/terraform-aws-opensearch.git

---

Código terraform aqui.

Variáveis (VPC)

# REGIAO DO PROVISIONAMENTO
variable "aws_region" {
  default = "us-east-2"
}

# PRIMEIRA ZONA DE DISPONIBILIDADE
variable "az1" {
  default = "us-east-2a"
}

# SEGUNDA ZONA DE DISPONIBILIDADE
variable "az2" {
  default = "us-east-2b"
}

# USAR AQUI A IDENTIFICACAO DO AMBIENTE
variable "customer_env" {
  default = "lab-o11y"
}

# INFORME O NOME QUE IRA UTILIZAR NO CLUSTER EKS # IMPORTANTE PARA RESOURCE K8S SHARING
variable "cluster_eks_name" {
  default = "lab-o11y"
}

# CIDR VPC
variable "vpc_cidr_block" {
  default = "172.29.0.0/16"
}

# SUBNET PUBLICA AZ1
variable "subnet_public_1_cidr" {
  default = "172.29.0.0/20"
}

# SUBNET PUBLICA AZ2
variable "subnet_public_2_cidr" {
  default = "172.29.16.0/20"
}

# SUBNET PRIVADA AZ1
variable "subnet_private_1_cidr" {
  default = "172.29.32.0/20"
}

# SUBNET PRIVADA AZ2
variable "subnet_private_2_cidr" {
  default = "172.29.48.0/20"
}

Variáveis (EKS)

# Project info

cluster_name = "lab-o11y"
project      = "o11y with otel and aos"
project_env  = "dev"
region       = "us-east-2"
k8s_version  = "1.24"

# VPC and Subnets

vpc_id         = "vpc-0c496e66aaff25697"
vpc_cidr_block = "172.29.0.0/16"

public_access_cidrs = [
  "0.0.0.0/0",
]

subnets = [ # Private subnets for eks cluster
  "subnet-05c30cc378277b402",
  "subnet-03e9fd8df45175919"
]

public_subnets = [ # Public subnets for nlb
  "subnet-09881eca61aeeb25a",
  "subnet-0300707fbef49d07d"
]

# Autoscaling configuration

instance_type = "t3.medium"
desired_size  = "2"
min_size      = "1"
max_size      = "4"

# Disk

disk_size = "30"

# Endpoint access

endpoint_private_access = false
endpoint_public_access  = true

# Logging

enabled_cluster_log_types = [
  "api",
  "audit",
  "controllerManager"
]

Variáveis (OpenSearch)

Abaixo apenas a variáveis que precisam ser informadas neste lab:

variable "cluster_name" {
  description = "The name of the OpenSearch cluster."
  type        = string
  default     = "lab-o11y"
}

variable "cluster_version" {
  description = "The version of OpenSearch to deploy."
  type        = string
  default     = "2.3"
}

variable "master_user" {
  description = "The master user of the OpenSearch cluster."
  type        = string
  default     = "workshop"
}

variable "master_password" {
  description = "The master password of the OpenSearch cluster."
  type        = string
  default     = "Workshop@2023"
}

Sequência de execução terraform:

1. VPC: https://github.com/paulofponciano/lab-microservices-o11y-amazon-opensearch/tree/main/vpc
2. EKS: https://github.com/paulofponciano/lab-microservices-o11y-amazon-opensearch/tree/main/eks
3. OpenSearch (fork): https://github.com/paulofponciano/lab-microservices-o11y-amazon-opensearch/tree/main/terraform-aws-opensearch/examples/minimal

Construção

• Stack VPC:

terraform init
terraform plan
terraform apply

Output:

• Stack EKS:

terraform init
terraform plan --var-file variables.tfvars
terraform apply --var-file variables.tfvars

Output:

• Stack OpenSearch:

terraform init
terraform plan
terraform apply

Não é preciso informar os dois inputs solicitados abaixo no apply:

Output (Console)

• Informe o domain endpoint, user e password do AOS no arquivo yaml que está em data-prepper.yaml — https://github.com/paulofponciano/lab-microservices-o11y-amazon-opensearch/blob/main/eks/samples/apps-ms/01-data-prepper/kubernetes/

• Conecte no cluster EKS:

aws eks --region us-east-2 update-kubeconfig --name lab-o11y

• Faça o deploy dos microsserviços executando o script app-deploy.sh — https://github.com/paulofponciano/lab-microservices-o11y-amazon-opensearch/tree/main/eks/samples/apps-ms:

chmod +x app-deploy.sh
./app-deploy.sh

• Acesse a aplicação do AWS workshop e gere dados — É possível utilizar a url do Network Load Balancer que foi criado na stack do EKS:

• Utilizando a url do Amazon OpenSearch Dashboards, faça login com user e password master informados na criação da stack OpenSearch:

• No OpenSearch Dashboards, agora é possível explorar os traces recebidos através do OpenTelemetry:

Observability — Trace analytics — Traces

• Crie um novo index pattern para os logs do kubernetes que são recebidos através do fluent-bit:

Stack Management — Index patterns

• Em 'Discover' é possível explorar os logs conforme o index pattern criado anteriormente:

Happy building!