DEV Community

John Ajera
John Ajera

Posted on

How to Configure Logging in Amazon EKS Fargate with Terraform

#aws #eks #terraform #cloudwatch

Amazon EKS on Fargate gives you serverless pods — but you still need logs!
This guide walks you through enabling logging visually and minimally using Terraform.

📘 Why This Matters

By default, Amazon EKS on Fargate doesn’t log application output. You must opt-in using Fluent Bit — a log router that reads from /var/log/containers and pushes logs to CloudWatch. All you need is:

  • EKS cluster using Fargate
  • Pod execution IAM role
  • Terraform

📦 Step 1: Create Logging Namespace 

resource "kubernetes_namespace" "aws_observability" {
  metadata {
    name = "aws-observability"
    labels = {
      "aws-observability" = "enabled"
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

📝 Step 2: Add Fluent Bit ConfigMap

Fluent Bit automatically detects this ConfigMap if it's named aws-logging and placed in the aws-observability namespace.

Make sure it contains valid [FILTER] and [OUTPUT] sections only. Avoid [INPUT] or [SERVICE] blocks — they are managed by AWS.

resource "kubernetes_config_map" "aws_logging" {
  metadata {
    name      = "aws-logging"
    namespace = kubernetes_namespace.aws_observability.metadata[0].name
  }

  data = {
    "filters.conf" = <<-EOT
      [FILTER]
          Name kubernetes
          Match logging-enabled.*
          Merge_Log On
          Keep_Log Off
    EOT

    "output.conf" = <<-EOT
      [OUTPUT]
          Name cloudwatch_logs
          Match logging-enabled.*
          region ap-southeast-1
          log_group_name /aws/eks/my-cluster/logs
          log_stream_prefix from-fluent-bit-
          auto_create_group true
    EOT
  }
}
Enter fullscreen mode Exit fullscreen mode

🏷️ Step 3: Annotate Pods to Enable Logging

Add the following annotations to opt your pod in for logging. These control whether Fluent Bit processes logs and which tag it uses for matching.

🧠 fluentbit.io/tag must match the Match pattern in your ConfigMap.

annotations = {
  "eks.amazonaws.com/enable-logging" = "true"
  "fluentbit.io/tag"                 = "logging-enabled.nginx"
}
Enter fullscreen mode Exit fullscreen mode

🔐 Step 4: IAM Role Policy 

resource "aws_iam_role_policy" "fargate_logging" {
  name = "EKSFargateLogging"
  role = aws_iam_role.fargate_execution.name

  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Sid      = "CreateLogGroup",
        Effect   = "Allow",
        Action   = "logs:CreateLogGroup",
        Resource = "arn:aws:logs:ap-southeast-2:123456789012:log-group:/aws/eks/my-cluster/logs"
      },
      {
        Sid    = "StreamAndPutLogs",
        Effect = "Allow",
        Action = [
          "logs:CreateLogStream",
          "logs:PutLogEvents",
          "logs:DescribeLogStreams"
        ],
        Resource = "arn:aws:logs:ap-southeast-2:123456789012:log-group:/aws/eks/my-cluster/logs:*"
      }
    ]
  })
}
Enter fullscreen mode Exit fullscreen mode

✅ Verify Logging

📌 Check pod logging status:

kubectl describe pod <pod-name>
Enter fullscreen mode Exit fullscreen mode

Look for:
Logging: Successfully enabled logging for pod

📌 Check CloudWatch Logs:
Go to: CloudWatch > Log groups > /aws/eks/my-cluster/logs

📌 List pods and tail logs:

kubectl get pods
kubectl logs <pod-name>
Enter fullscreen mode Exit fullscreen mode

📌 Check pod status:

kubectl describe pod <pod-name>
Enter fullscreen mode Exit fullscreen mode

Look for:
Logging: Successfully enabled logging for pod

📌 View logs:

  • CloudWatch: /aws/eks/my-cluster/logs
  • CLI: kubectl logs <pod-name>

🧪 Troubleshooting

  • ✅ ConfigMap must be < 5300 characters
  • ✅ Use only [FILTER], [OUTPUT], [PARSER]
  • 🔁 Restart pods for config changes to apply

🎯 Done

With logging set up per pod using Terraform, you now have fine-grained control and visibility into your EKS Fargate workloads.

Minimal config. Maximum observability. 💡

Top comments (0)