ECS Orchestration Part 2: Service to Service comunication

The first post in the series was about choosing the correct network type for running containers on ECS (Part 1), and in this post we continue with a network-related topic, answering at the following question:
"How can I make the services within the ECS cluster communicate with each other?"
Service-to-service communication refers to the exchange of data and messages between different microservices or containerized applications. In a microservices architecture, different parts of an application are broken down into smaller, more modular services that communicate with each other via APIs. This allows for greater flexibility, scalability, and agility when developing and deploying applications.
For microservices to function properly, they need to be able to communicate with each other reliably and securely. This is where ECS comes in.
ECS offers several ways to enable service-to-service communication in microservices, let us explore the main ways.

Amazon ECS Service Connect

ECS Service Connect uses a proxy container as a sidecar for each activity within your service. The proxy container intercepts outgoing connections and redirects them to the IP address of the requested service.

Key Features:

• Integrated Service Discovery: Like ECS Service Discovery, it allows services to discover each other via DNS. However, Service Connect also adds load balancing and traffic management.
• Simplified Networking: Abstracts the complexity of service-to-service communication by allowing services to use DNS names like service-name.namespace without managing individual IPs or load balancers.
• Built-in Load Balancing: Automatically balances traffic between service tasks, eliminating the need for manually configuring AWS Load Balancers.
• Automatic Encryption: Traffic between services is encrypted by default, providing a secure communication layer.
• Cross-Cluster Communication: Services can communicate across different ECS clusters.
• No Additional Infrastructure: Unlike ECS Service Discovery, you don’t need to configure and manage load balancers manually. Service Connect handles this behind the scenes.
• Service Mesh: ECS Service Connect introduces service mesh-like capabilities by enabling traffic management, retries, timeouts, and load balancing automatically.

Use Cases:

• You want automated load balancing and traffic management for ECS services.
• You need simplified service communication and don’t want to manage AWS Load Balancers or Route 53 configurations manually.
• Encryption of service-to-service communication is required out of the box.
• Cross-cluster communication between services is needed, and you want to avoid manual configuration.
• Ideal for microservices architectures where services need dynamic communication, load balancing, and secure interconnections.

Pricing Model

• No additional charges for Service Connect itself: As of now, ECS Service Connect does not incur extra costs beyond the standard ECS service and task pricing. You pay for the underlying ECS resources (like EC2 instances or Fargate) that you are using.
• AWS Data Transfer Charges: You may incur data transfer charges if your services are communicating across different AWS regions or accounts, but intra-region communication is typically free.
• CloudWatch Costs: If you enable CloudWatch metrics for your ECS tasks/services, there may be additional charges based on your monitoring needs.

Key Points

• No separate pricing for Service Connect; it is included with the ECS service.
• Simplified billing as it integrates seamlessly into your existing ECS pricing model.

If you what to test Service connect with terraform, you can see the example below:

Change your ECS service confinguartion with this example.

resource "aws_ecs_service" "nginx_service" {
  name            = "nginx-service"
  cluster         = aws_ecs_cluster.example_cluster.id
  task_definition = aws_ecs_task_definition.nginx_task.arn
  desired_count   = 1
  launch_type     = "FARGATE"

  network_configuration {
    subnets          = aws_subnet.public[*].id
    security_groups  = [aws_security_group.ecs_service_sg.id]
    assign_public_ip = true
  }

  service_connect_configuration {
    enabled = true

    namespace = "example.local"  # This is the Cloud Map namespace for service discovery

    service {
      port_name = "http"
      discovery_name = "nginx-service"
      client_aliases {
        port = 80
        dns_name = "nginx-service.example.local"  # DNS name for service discovery
      }
    }
  }
}

Verifying Service Connect

nslookup nginx-service.example.local

Amazon ECS Service Discovery

Service Discovery in Amazon ECS allows services within a cluster to communicate with each other by name, without needing to know the IP addresses or ports of the services. This is especially useful in dynamic environments where containers can come and go, and their IP addresses may change.
Amazon ECS integrates with AWS Cloud Map and Route 53 for service discovery, allowing services to register and look up DNS names dynamically.

Key Features:

• AWS Cloud Map Integration: ECS Service Discovery integrates with AWS Cloud Map, registering ECS services with a DNS-based service registry.
• DNS-based Discovery: Services discover other services by querying DNS records. ECS tasks are registered with A or SRV DNS records.
• Internal Communication: Primarily used for communication within the VPC (via private DNS namespaces), though it can also support public namespaces.
• Health Checks: Can use Route 53 health checks to ensure traffic is routed only to healthy instances.
• Manual Load Balancing: Requires configuring your own load balancers, or you can use AWS Application/Network Load Balancers with ECS tasks.

Use Cases:

• Simple service discovery when you have multiple services that need to find each other based on DNS names.
• If you are already using AWS Cloud Map for service discovery across AWS services.
• For scenarios where manual control over load balancing and routing policies is needed.

Pricing Model

• Service Discovery Charges: You are charged based on the number of API calls made and the number of service instances registered in Cloud Map.
• Service Instances: You incur costs for each service instance registered in Cloud Map (a small monthly fee per instance).
• API Calls: Charges are applied for API calls made to AWS Cloud Map for operations like registering, deregistering, and discovering services.
• Health Checks: If you configure health checks for services, there may be additional costs based on the frequency of checks and the number of checks performed.

Key Points

• Variable costs based on the number of services, instances, and API calls made to Cloud Map.
• More granular control and customization options compared to Service Connect, but at an additional cost.

The following example show you how can setup a Service Discovery using ECS and terraform:

resource "aws_service_discovery_private_dns_namespace" "cloudmap_namespace" {
  name        = "myapp.local"
  description = "Private namespace for my ECS services"
  vpc         = aws_vpc.main.id
}

resource "aws_service_discovery_service" "myapp_service" {
  name = "nginx"

  dns_config {
    namespace_id = aws_service_discovery_private_dns_namespace.cloudmap_namespace.id
    dns_records {
      type = "A"
      ttl  = 60
    }
    routing_policy = "MULTIVALUE"
  }

  health_check_custom_config {
    failure_threshold = 1
  }
}

resource "aws_ecs_service" "ecs_service" {
  name            = "nginx-service"
  cluster         = aws_ecs_cluster.ecs_cluster.id
  task_definition = aws_ecs_task_definition.nginx_task.arn
  desired_count   = 1
  launch_type     = "FARGATE"

  network_configuration {
    subnets          = aws_subnet.subnet[*].id
    security_groups  = [aws_security_group.ecs_service_sg.id]
    assign_public_ip = true
  }

  service_registries {
    registry_arn = aws_service_discovery_service.myapp_service.arn
  }
}

Verify Service Discovery
Once your ECS service is up and running, you can verify the service discovery by resolving the DNS name of the service within the same VPC. For example:

nslookup nginx.myapp.local