Exploring DNS for Multi-Account Hybrid Cloud Setup

When migrating to cloud, enterprises often have internal business applications spread across on-premise and multiple AWS accounts belonging to various business units. Such scenarios require consistent DNS records and domain names between different AWS accounts and on-premises.

In this article we will explore AWS service and architecture to achieve this use case.

We know AWS has native DNS service called Route53. This DNS service has always been associated with VPC and often also known by several other names like:-

1. Amazon Provided DNS
2. VPC Resolver
3. +2 Resolver
4. .2 Resolver
5. EC2 DNS Resolver

however the problem is that Route 53 does not respond to queries which are not originating from within VPC. This essentially means that even there is an established connectivity between on-prem and cloud via VPN or direct connect, on-prem DNS is unable to talk to Route 53.

One solution can be to have a custom DNS server hosted on EC2 within VPC which can then connect to On-prem DNS and resolve. However It is a setup and management overhead like EC2 setup, high availability considerations, vulnerability management and so on.

AWS has another managed service called route 53 resolver inbound/outbound endpoints which can be configured to take care of all required activities to connect on-prem DNS to AWS cloud. The advantage is that it is a managed service without any operational overhead. There are just endpoints and forwarding rules to be configured.

Architecture is explained below:-

• Instances within a VPC will use the Route 53 Resolver (Amazon Provided DNS).
• Private hosted zones will be associated with shared services VPC.
• These Private hosted zones will also be associated with other VPCs in the environment if required.
• Conditional forward rule(s) from the on-premises DNS servers will have an inbound Route 53 Resolver endpoint as their destination.
• Rule(s) for on-premises domain names are created that leverage an outbound Route 53 Resolver endpoint.

Double clicking on Inbound resolver endpoint:-

• The inbound endpoints should be deployed in 2 availability zones for high availability.
• Inbound resolver endpoint will creates routable ENIs in VPC reachable over AWS Direct Connect or VPN.
• All the internal domains will be associated with DNS VPC in shared service account.
• Nomenclature: one “endpoint” == multiple ENIs. Limit: 10,000 QPS per ENI

On-Prem DNS needs to be configured to enable forwarding of queries Route 53 inbound endpoints. These forwarding rules should point to inbound IP addresses.

Below inputs are required to create inbound endpoints.

Double clicking on Outbound resolver endpoint:-

To resolve domains hosted at on-prem, Route53 Outbound endpoints need to be deployed.

These endpoints will serve as the path through which all queries will be forwarded out of the VPC (towards on-prem DNS server).

Outbound endpoints will be directly attached to the DNS VPC in shared service account and indirectly associated with other VPCs via rules. i.e, if a forwarding rule is shared with VPC that does not own the outbound endpoint, all queries that match the forwarding rule pass through to the DNS VPC and then forward out.

The outbound endpoint may reside in an entirely different Availability Zone than the VPC that originally sent the query, and there is potential for an Availability Zone outage in the DNS VPC to impact query resolution in the VPC using the forwarding rule. Therefore, it is recommended to deploy outbound endpoints in multiple Availability Zones to avoid any impact due to AZ outage.

Outbound resolver endpoints are deployed using three separate resources as mentioned below:

• An endpoint, similar to the inbound resolver with two or more IP addresses and a security group. Restrict access to only designated target resolvers. Update network ACLs and routing tables.
• A rule, which specifies the domain to conditionally forward to name servers.
• An association, which links the rule to one or more VPCs

Forwarding Rules.

• System and forward rules define the path
• Forward all public DNS resolution via route 53 resolver
• Route 53 Resolver should answer: amazonaws.com.
• Private Hosted Zone: abc.mycloud.com.
• Corp office namespace: corp.enterprise.com
• VPC CIDR: 10.10.0.0/23
• On-premises CIDR range: 10.20.0.0/23

Resolver Rule summary

• Most specific rule wins
• Private DNS, PrivateLink endpoints, and VPC DNS get auto defined rules
• They can be overridden by Forward rule
• Best practice to allow SYSTEM resolve amazonaws.com.
• Need to create reverse records, e.g., for Kerberos
• VPC CIDR ranges get /24 rules (e.g., x.y.10.in-addr.arpa) auto defined.

Lets look at the sample terraform configuration files to configure these Inbound and Outbound resolver with forwarding rules.

Below terraform script with create all the required resources in Central DNS account (many times also referred as shared service account)

Terraform script to setup Central-DNS-VPC

resource "aws_vpc" "CentralDNS" {
  cidr_block       = "172.219.244.128/27"
  instance_tenancy = "default"
  enable_dns_support = true
  enable_dns_hostnames = true

  tags = {
    Name = "CentralDNS"
  }
}


output "aws_vpc" {
  value = "aws_vpc.CentralDNS.ID"
}

Terraform script for Subnet-1

resource "aws_subnet" "DNSPvtSub1" {
  vpc_id     = aws_vpc.CentralDNS.id
  cidr_block = "172.219.244.128/28"


  tags = {
    Name = "DNSPvtSub1"
  }
}

Terraform script for Subnet-2

resource "aws_subnet" "DNSPvtSub2" {
  vpc_id     = aws_vpc.CentralDNS.id
  cidr_block = "172.219.244.144/28"


  tags = {
    Name = "DNSPvtSub2"
  }
}

Terraform script for route table

resource "aws_route_table" "RT-PVTSubnets" {
  vpc_id = aws_vpc.CentralDNS.id


  tags = {
    Name = "RT-PVTSubnets"
  }
}

Terraform script for Route-Table-Association

resource "aws_route_table_association" "RT-Association-PVTSubnet1" {
  subnet_id      = aws_subnet.DNSPvtSub1.id
  route_table_id = aws_route_table.RT-PVTSubnets.id
}


resource "aws_route_table_association" "RT-Association-PVTSubnet2" {
  subnet_id      = aws_subnet.DNSPvtSub2.id
  route_table_id = aws_route_table.RT-PVTSubnets.id
}

Terraform script for Security-Group

resource "aws_security_group" "SG-DNS" {
  name        = "SG-DNS"
  description = "Allow DNS traffic"
  vpc_id      = aws_vpc.CentralDNS.id



  ingress {
    from_port   = 172
    to_port     = 172
    protocol    = "tcp"
    cidr_blocks = [var.CIDR]
  }


  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "tcp"
    cidr_blocks = [var.allowall]
  }


  tags = {
    Name = "DNS_SG"
  }
}

Terraform script for DNS Inbound Endpoint

resource "aws_route172_resolver_endpoint" "Inbound-EP" {
  name      = "Inbound-EP"
  direction = "INBOUND"


  security_group_ids = [
    aws_security_group.SG-DNS.id        
  ]


  ip_address {
    subnet_id = aws_subnet.DNSPvtSub1.id
    ip = var.InboundEP1["ap-northeast-1]"
  }


  ip_address {
    subnet_id = aws_subnet.DNSPvtSub2.id
    ip        = var.InboundEP2
  }


  tags = {
    Name = "Inbound-EP"
  }
}

Terraform Script for DNS outbound endpoint

resource "aws_route172_resolver_endpoint" "Outbound-EP" {
  name      = "Outbound-EP"
  direction = "OUTBOUND"


  security_group_ids = [
    aws_security_group.SG-DNS.id        
  ]


  ip_address {
    subnet_id = aws_subnet.DNSPvtSub1.id
    ip = var.OutboundEP1
  }


  ip_address {
    subnet_id = aws_subnet.DNSPvtSub2.id
    ip        = var.OutboundEP2
  }


  tags = {
    Name = "Outbound-EP"
  }
}

Terraform script for forwarding Rule-For-On-prem

resource "aws_route172_resolver_rule" "fwd-rule-to-on-Prem" {
  domain_name          = "exampleintra.net"
  name                 = "fwd-rule-to-on-Prem"
  rule_type            = "FORWARD"
  resolver_endpoint_id = aws_route172_resolver_endpoint.Outbound-EP.id


  target_ip {
    ip = var.onpremdns-firstip
  }


  target_ip {
    ip = var.onpremdns-secondip
  }


  tags = {
    Name = "fwd-rule-to-on-Prem"
  }
}




#FW-Rule-For-Cloud 
resource "aws_route172_resolver_rule" "fwd-rule-to-cloud" {
  domain_name          = "ec1-aws.cloud.exampleintra.net"
  name                 = "fwd-rule-to-cloud"
  rule_type            = "FORWARD"
  resolver_endpoint_id = aws_route172_resolver_endpoint.Outbound-EP.id


  target_ip {
    ip = var.InboundEP1    
  }


  target_ip {
    ip = var.InboundEP2   
  }


  tags = {
    Name = "fwd-rule-to-cloud"
  }
}

Terraform script for Rule-Association

resource "aws_route172_resolver_rule_association" "rule-association-onprem" {
  resolver_rule_id = aws_route172_resolver_rule.fwd-rule-to-on-Prem.id
  vpc_id           = aws_vpc.CentralDNS.id
}

Terraform script for Rule-Sharing

resource "aws_ram_resource_share" "FWD-rule-share" {  
  name = "FWD-rule-share"
  allow_external_principals = false 
}


resource "aws_ram_principal_association" "FWD-rule-principal-association1" {
  principal          = var.WLacc1  
  resource_share_arn = aws_ram_resource_share.FWD-rule-share.arn
}


resource "aws_ram_principal_association" "FWD-rule-principal-association2" {
  principal          = var.WLacc2  
  resource_share_arn = aws_ram_resource_share.FWD-rule-share.arn
}


resource "aws_ram_resource_association" "FWD-resource-association-onprem" {
  resource_arn       = aws_route172_resolver_rule.fwd-rule-to-on-Prem.arn  
  resource_share_arn = aws_ram_resource_share.FWD-rule-share.arn
}


resource "aws_ram_resource_association" "FWD-resource-association-cloud" {  
  resource_arn       = aws_route172_resolver_rule.fwd-rule-to-cloud.arn
  resource_share_arn = aws_ram_resource_share.FWD-rule-share.arn
}

Hosted-Zone

resource "aws_route172_zone" "Frankfurt-PHZ" {
  name = var.phz


  vpc {
    vpc_id = aws_vpc.CentralDNS.id
  }
}




#Authorization


data "aws_region" "current" {}


/*
resource "aws_route172_zone" "domain_example" {
  name = "example.com"
}
*/


resource "null_resource" "create_remote_zone_auth" {
  count = "${length(var.accounts_to_auth)}"
  triggers = {
    zone_id = "${aws_route172_zone.Frankfurt-PHZ.zone_id}"
  }


  provisioner "local-exec" {
    command = "aws route172 create-vpc-association-authorization --hosted-zone-id ${aws_route172_zone.Frankfurt-PHZ.zone_id} --vpc VPCRegion=${data.aws_region.current.name},VPCId=${element(var.accounts_to_auth, count.index)}"
  }
}

Below is the Terraform variables file.

Variable-File

variable "CIDR" {}
variable "allowall" {}


variable "InboundEP1" {
  type = map
  default = {
    eu-central-1  = ["172.219.40.133"]
    eu-west-1 = ["172.219.44.133"]
    us-west-2 = ["172.219.140.133"]
    us-east-1 = ["172.219.144.133"]
    sa-east-1 = ["172.219.156.133"]
    ap-southeast-1  = ["172.219.244.133"]
    ap-northeast-1  = ["172.219.248.133"]
  }
}



variable "InboundEP2" {}
variable "OutboundEP1" {}
variable "OutboundEP2" {}
variable "onpremdns-firstip" {}
variable "onpremdns-secondip" {}
variable "WLacc1" {}
variable "WLacc2" {}
variable "phz" {}
variable "accounts_to_auth" {
  default = [
    "vpc-0b13f9b7a557f0568"
  ]
}

Terraform.tfvar file

CIDR = "172.0.0.0/8"
allowall  = "0.0.0.0/0"


InboundEP2 = "172.219.244.150"
OutboundEP1 = "172.219.244.134"
OutboundEP2 = "172.219.244.151"
onpremdns-firstip = "172.0.172.6"
onpremdns-secondip = "172.0.172.4"
WLacc1 = "111111111111"
WLacc2 = "222222222222"
phz = "ec1-tafaws.cloud.exampleintra.net"s

*Conclusion *- In this article we explored the scenerio and implementation of Hybrid DNS on AWS where requirement is to have consistent DNS records and domain names between different AWS accounts and on-premises.

Reference- Amazon web service