AWS PrivateLink for more secure and cost-effective inter-service communication

Naween Fonseka — Sat, 04 Sep 2021 14:00:01 +0000

In this article, I'll be explaining how to use AWS PrivateLink to secure and more cost-effective AWS inter-service communication. AWS PrivateLink is a widely discussed area in AWS cloud and, yet I have decided to write my recent experience with AWS PrivateLink.

AWS PrivateLink

AWS PrivateLink was launched in 2019 to allow users to securely access AWS services without exposing their traffic to the public internet.

AWS PrivateLink is a highly available, scalable technology that enables you to privately connect your VPC to supported AWS services, services hosted by other AWS accounts (VPC endpoint services), and supported AWS Marketplace partner services.

Please find the AWS PrivateLink whitepaper from here.

The following figure describes the usage of PrivateLink.

As per the above figure, the diagram on the left shows ECS is pulling images from ECR using the public internet even though they are in the same AWS account. AWS PrivateLink, on the other hand, keeps communication with ECS and ECR within the AWS network, making it more secure and cost-effective.

AWS PrivateLink supports both same region and cross-region access. However, For cross-region PrivateLink to work, you may need to use inter-region VPC peering. In this article, I will only discuss the use of PrivateLink in the same region. More information about cross-region PrivateLink support may be found here.

VPC Endpoints

VPC endpoint allows AWS users to connect to AWS services and VPC endpoint services in a secure manner (services hosted by AWS customers and partners are known as VPC endpoint services). AWS PrivateLink is used to power VPC endpoints. AWS PrivateLink uses private IP addresses to communicate with internal AWS services. VPC Endpoints are categorized into three categories: (1) Interface Endpoints,(2) Gateway Load Balancer Endpoints and (3) Gateway Endpoints.

Interface Endpoints

AWS PrivateLink connects services provided by AWS using Interface Endpoints (Interface VPC Endpoints). AWS uses Network Load Balancers (NLB) to link interface endpoints to AWS services in the background while utilizing AWS PrivateLink.

Interface endpoints can be used to connect AWS managed services, customer hosted AWS Services and endpoint services.

Gateway Load Balancer Endpoints (GWLB Endpoints)

GWLB Endpoints are primarily used to intercept network traffic for security inspection. Only services that are configured to use Gateway Load Balancers are supported by GWLB Endpoints.

Gateway Endpoints

These types of endpoints are used as a route table target for traffic generated into AWS supported services. At the time of writing this article, only Amazon S3 and DynamoDB are supported for use of Gateway Endpoints.

Differences between GWLB Endpoints and Gateway Endpoints can be found here.

Please note that not all AWS services are supported by PrivateLink, and I'll only go through a handful that can utilize PrivateLink to secure and cost-effectively communicate with other services. Please find this link which will be useful in finding services that are compatible with PrivateLink.

Endpoint for ECR

Amazon ECR is a fully managed container registry that works with EKS, ECS, and Lambda. Please find the following steps to create a PrivateLink for ECR service. Without configuring a PrivateLink for ECR, you have been pulling the images from ECR to EKS/ECS or lambda via public internet by leaving the AWS Network. This comes as an added cost if you are using NAT Gateway to communicate with the public internet.

For said services to pull images in ECR privately, we may need to create two endpoints.

Interface Endpoint for ECR Docker
Gateway Endpoint for Amazon S3

Let's begin!

Navigate to the VPC service in AWS Console and create an endpoint using the Create Endpoint button.

Leave the Service category as default 'AWS Services' and search for 'ecr' under service name. Select Interface com.amazonaws.[region].ecr.dkr. (select the region you are currently working on)

Select the desired VPC and subnets where your instances reside. Leave the Enable DNS name as ticked.

Create a new dedicated security group and allow all inbound HTTPS traffic from port 443. This will allow incoming traffic from the subnets of the selected VPC.

Search and select the newly created security group while keeping the default policy as it is. Create a new tag with Key as Name and Value as ECR_endpoint or something you prefer and save the endpoint.

Initially, the endpoint will be pending.

The status should change to accessible in a few minutes if everything goes well.

Endpoint for S3

As mentioned earlier, for ECR to work privately, we may need to create a Gateway Endpoint for Amazon S3 as ECR uses S3 to store docker images as layers under the hood.

Same as before. create a new endpoint and search for com.amazonaws.[region].s3. (select the region you are currently working on).

Select the VPC where your instances reside and select the appropriate route table attached to the VPC. Leave the rest as default and save the endpoint.

Voilà! Both endpoints should be in available state after a while.

That should do it for ECR. If everything went well, you should experience a sudden change in the NAT gateways metrics (in Cloudawatch). Furthermore, in AWS Cost Explorer, do notice the cost change for EC2-Other ($) (Cost change might take some time, therefore notice the cost change after few days/weeks).

There are few other services that can help reduce the cost and enhance AWS security.

Endpoint for Cloudwatch Logs

When adding logs to the Cloudwatch by services such as ECS/EKS and Lambda, there are sent to the Cloudwatch through the public internet. This increases both the security risk for sensitive data and NAT Gateway cost.

To avoid the above, let's create an Interface Gateway for Cloudwatch Logs.

Head over to the Endpoints and create a new interface endpoint using com.amazonaws.[region].logs select the region you are currently working on.

Select the desired VPC and subnets where your instances reside.

Create a security group same as before by allowing HTTPS traffic through port 443 and save the endpoint.

Now your logs should be added to Cloudwatch using the internal AWS network.

Endpoint for SSM

There's one more service I would like to highlight and that is AWS SSM. This can be useful if you are having an EC2 instance without access to the public internet. For example, by creating an Interface Endpoint for SSM, your instance can securely access parameters in the Parameter Store without internet access.

Let's create a new VPC endpoint using the com.amazonaws.[region].ssm service. Region should be same as your current selected region in the AWS console.

Same as before, create a new security group for SSM Privatelink with allowing HTTPS access through port 443. Keep the rest the same as before and save the endpoint.

Finally, you should have four VPC endpoints in available state contributing secure inter-service communication to your resources.

AWS PrivateLink with Cross Account

You may be wondering how to use AWS PrivateLink when you have multiple AWS accounts for different environments and, there can be resources that are commonly used by other accounts. For example, all your ECR images could be in one account(Let's say master/root account) and, other accounts such as Staging/Production are using the master/account's ECR to store and pull the images.

In that case, creating a PrivateLink is similar to the same account PrivateLink creation. PrivateLinks needs to be created in each account that is trying to pull the images from.
Please find the diagram I have created which illustrates PrivateLink usage with multi accounts.

As per the above diagram, secure cross account access with PrivateLink can be achieved by creating VPC endpoints in each account(same region) where you are trying to access the resources (Make sure to avoid creating endpoints in the root/master account).

References

AWS PrivateLink Whitepaper (https://d1.awsstatic.com/whitepapers/aws-privatelink.pdf)
AWS PrivateLink and VPC endpoints (https://docs.aws.amazon.com/vpc/latest/privatelink/endpoint-services-overview.html)
Cover photo by KAL VISUALS on Unsplash

DEV Community: Naween Fonseka