How to Solve Overlapping IP Addresses in AWS

Recently, I got a chance to present at the Auckland AWS Community meetup to talk about "How to Solve Overlapping IP Addresses in AWS". PDF slide deck is here.

Every service, transaction, and everything you do with AWS or any public cloud uses networking. But it's quite common to see network and security design decisions made afterthoughts, which has led to overlapping IP addresses issues. But there are times when overlapping IP can't be avoided as well.

In this post I'm going to share what those overlapping IP scenarios are, how we can solve the issue, and I hope everyone will find this helpful.

Overlapping IP Address Scenarios

When organizations are starting to deploy services in the cloud and spin up VPCs, it's easy to fall into overlapping IPs between VPCs or with on-premises. There are a few common use cases where private networks with overlapping IP addresses need to be connected.

Connecting default VPCs. This may not necessarily be your VPC, but I've seen multiple organisations using the default VPC and they need to connect the two which is not possible out of the box.

It's still possible for multiple VPC's to end up on the same CIDR block (e.g., 10.0.0.0/16, 192.168.0.0/16) whether it's within AWS or connecting another cloud provider. The reason could be mergers & acquisitions or connecting to other networks like third parties, partners, B2Bs.

The most common scenario is when an organisation has cloud services and wants to deliver a service to a customer or partner using private IP addresses. They won't have the ability to ask their clients to change their IP space.

Solution Options to Overlapping IP Addresses

During the meetup event, I discussed at least three possible solutions to this issue. This post will cover them at a high level, I may have to write separate posts to cover them in more detail.

1 | AWS Private Link

AWS PrivateLink service has been around for a while, and it allows you to publish services from one VPC that you can access from another VPC. Through VPC Endpoint (VPCE), you can extend services behind an NLB (Network Load Balancer) into a different VPC. PrivateLink is made up of NLB and VPCE. Few things to note, Private Link is uni-directional and only works with TCP traffic

Here's a sample terraform code that creates VPCs with overlapping IPs and connects them using Private Link: https://github.com/bayupw/terraform-aws-overlapping-private-link-demo

2 | AWS NAT Gateway

Use AWS NAT Gateway if PrivateLink isn't a feasible option, especially when you're having UDP traffic.

To use AWS NAT Gateway, the services need to be created behind an ALB and published over a separate secondary VPC CIDR block that doesn't overlap. The secondary VPC CIDR blocks can then be connected using AWS Transit Gateway. Be sure to disable the route propagation as you don't want to advertise the overlapping IP spaces. One requirement for this solution is that the NAT Gateway needs to be deployed on the consumer VPC, which may not be feasible for some organisations. There's a blog post that covers it in detail here: https://aws.amazon.com/blogs/networking-and-content-delivery/how-to-solve-private-ip-exhaustion-with-private-nat-solution/

Here's an example terraform code for deploying this solution: https://github.com/bayupw/terraform-aws-overlapping-nat-gateway-demo

3 | Aviatrix Secure Cloud Networking Platform

Another option is is to use Aviatrix to solve this overlapping IP issues. With Aviatrix gateways, you can solve overlapping IP addresses using an intelligent, flexible NAT solution. The Aviatrix Controller configures the routes to the AWS VPC route tables based on the Virtual CIDRs configuration. With this solution, all the NAT is done on the provider's end, so the consumer doesn't have to configure any complicated NAT.

Here's a sample terraform code to deploy the solution: https://github.com/bayupw/terraform-aviatrix-aws-overlapping-demo

There's also a nicely written blog post written by Brad Hedlund and AWS Solution Architects which covers this solution in detail from starting up the Aviatrix Controller, deploying the Aviatrix Gateways, configuring intelligent NAT, testing the traffic, and observing the traffic flow through Aviatrix CoPilot: https://aws.amazon.com/blogs/awsmarketplace/how-to-solve-overlapping-ip-addresses-using-the-aviatrix-cloud-network-platform/

Besides solving overlapping IP issues, Aviatrix also provides additional features that organizations often need in an overlapping IP scenario, like the option for bi-directional traffic, policy-based segmentation, traffic inspection via Next-Generation Firewall and visibility of traffic flow regardless of the cloud provider. There was an Aviatrix TechTalk session which covered this overlapping scenario in a SaaS provider scenario with a nice demo.

Alternatively, you can read about the solution here: https://aviatrix.com/resources/all-content/aviatrix-validated-design-saas-providers-infrastructure

Summary

Those are the three options to solve overlapping IPs issues.

1. AWS PrivateLink: use this if you're only dealing with uni-directional TCP traffic
2. AWS NAT Gateway: if PrivateLink isn't an option, you need UDP traffic and perhaps need to steer the traffic to a Security VPC, you can evaluate AWS NAT Gateway for hiding the overlapping IP space.
3. Aviatrix: last but not least, if none of the other options work for you or you need extra features like visibility, network segmentation, policy-based traffic steering to a Next-Gen Firewall, which often required in a production enterprise network, I'd suggest exploring Aviatrix! 🙂

I hope you found this article is informative and feel free to reach out if you have any questions!