Restructuring NAT Gateway Usage and replacing it with Squid Proxy

We generally deploy our instances in a private network within our VPC in order to securely access them. However, for these instances, we require an outbound connection in order to have the latest OS security patches, and sometimes even some applications might need to connect to third-party URLs for various reasons. To have access to the internet we more often use cloud-provided services like NAT Gateway.

The below diagram depicts the typical deployment and usage of NAT Gateway

Nat gateway is highly available and takes care of most of the responsibility in terms of managing the service. But it becomes complex when organizations have several AWS accounts and then NAT gateway is deployed and managed in each of these accounts

Below are the few disadvantages with respect to use the Nat Gateway

Cost increases with the number of NAT Gateways deployed in each account
There is no ruleset to allow or deny traffic based on the rule list. This also includes access to malicious websites.
Detailed logging is missing.
Logs are not collected at a centralized location and no central control over them.

Solution Used

A centralized proxy solution is used to overcome the above-mentioned challenges
Squid Proxy(Open Source) is used for the outbound connection which gives more control at a centralized place
The solution is highly available, scalable, and is supported by different tenants.
This will save costs and operational effort while reducing the number of NAT gateways and the need for an Outbound proxy for each and every account.

This proxy will be deployed in a centralized account with a dedicated VPC. In order to have the Proxy VPC reachable from each tenant, VPC Interface endpoints powered by AWS private link will be used. Routing inside each VPC may be required in order to reach the VPC endpoint which will in fact be an ENI on a specific subset of the VPC.

To build this infrastructure solution. I have used terraform. In my opinion, terraform gives you more flexibility by writing less code compared to cloud formation.

Followed the concept of modules which gives more readability and easy understanding to the code. These modules can easily be reused. This also helps in reducing duplication, enable isolation, and enhance testability.

To over this challenge, SQUID Proxy is one of the possible ways to overcome

SQUID proxy is an open-source that can be used easily.
This proxy provides the filtering mechanism by which can easily restrict IPs and other websites.

This solution is designed to achieve high -availably using the autoscaling group with Network Load balancer.

The below is the structure for the Terraform modules for this use-case.

We can create different separations for each environment. This gives more control over the entire management of the infrastructure. With this approach, we can easily define have had lights configuration(Instance Type, high-availability) for dev and acceptance compared to the production environment.

One of the major challenges which we faced while building this solution was how we can consume the existing VPC and subnets which are part of the account itself. We wanted to avoid the hardcoding of the exiting VPC and subnets under variables files. For this, we use the filtering mechanism of the terraform. Based on the tag name we can easily search the desired VPC name and it’s respective subnets.