Amazon VPC Lattice — feasibility study

Amazon VPC Lattice has now become generally available (March 2023) and finally, I managed to give it a try and see whether it would meet the expectations it had aroused back at AWS re:Invent 2022. There were quite a few of them, e.g. having the ability to avoid using VPC peerings or VPC service endpoints to facilitate cross-account, cross-VPC applications communication while separating the core networking management from individual services configuration across the estate, as well as easily defining and attaching services to a wider application networking mesh.

Just in case…

“Amazon VPC Lattice is a fully managed application networking service that you use to connect, secure, and monitor all of your services across multiple accounts and virtual private clouds (VPCs).” ~ AWS

Need to know more? — check it out here since the rest of this story assumes you know the basics and is all about presenting the results of my early experimentation with VPC Lattice as well as sharing my findings and opinions.

Proof of concept

The idea was simple — I wanted to test out VPC Lattice functional capabilities as much as possible with just a light touch on the non-functional ones. And so I ended up building this…

What you can see above is a set-up containing:

• two AWS accounts being a part of the same AWS organization
• two VPC Lattice service networks where Test #1 is shared through RAM
• two VPC Lattice services where test-svc-1 is shared through RAM for association with the Test #2 service network
• three VPCs associated with service networks where VPC .103 is client-only
• test-svc-1 service with several different target types

Note: those Lambda-powered curlers in all three VPCs are there to facilitate sending custom HTTP requests to any VPC Lattice service for testing purposes. Need that Lambda function source code to try it yourself? It’s not sophisticated but does the job and here it is:

import json
import http.client
import ssl
from urllib.parse import urljoin

###################################################################################
REQ_PROTOCOL = '<???>'                            # options: 'http', 'https'
REQ_HOST     = '<???>'                            # e.g. myservice.mydomain.aws
REQ_PATH     = '/'                                # e.g. '/lambda80', '/lambda443'
REQ_HEADERS  = {                                  # a map of custom headers
    # "My-Header": "lambda-hh",                       
    "Content-Type": "application/json"
}
###################################################################################

def get(protocol, host, path, headers, event):
    """GET request"""

    if protocol == "https":
        conn = http.client.HTTPSConnection(host, context = ssl._create_unverified_context())
    elif protocol == "http":
        conn = http.client.HTTPConnection(host)
    else:
        return "[ERR] Unknown protocol provided!"

    try:
        conn.request('GET', path, json.dumps(event), headers)
        res = conn.getresponse()
        location_header = res.getheader("location")

        if location_header is not None:
            location = urljoin(path, location_header)
            # print(location)
            return get(protocol, host, location, headers, event)

        data = res.read()
    except Exception as error:
        return f"[ERR] {str(error)}"

    return data

def lambda_handler(event, context):
    """Lambda handler"""

    response = get(
        REQ_PROTOCOL,
        REQ_HOST,
        REQ_PATH,
        REQ_HEADERS,
        event
    )

    return {
        "statusCode": 200,
        "body": response,
        "headers": {
            "Content-Type": "application/json"
        }
    }

VPC Lattice service

That test-svc-1 is the main element though and most of the focus was put on the VPC Lattice service, service networks configuration aspects as well as cross-account and cross-VPC communication to services. It exposes privately several microservices under different combinations of protocols, ports, and paths represented by various AWS compute services and configured with the following target groups (TGs):

• EC2 in ASG
• ALB with EC2 in ASG
• ALB with ECS powered by Fargate
• Lambda functions

Apart from one Lambda function, everything else runs in private subnets across multiple availability zones.

Guess what!? It all worked very nicely!
But hey! Doesn’t that configuration and its elements look any familiar?
To me, it did hence I’m going to risk the following statement…

Amazon VPC Lattice service is an implementation of a private Application Load Balancer with cross-account- and auth-related features in mind.

VPC Lattice service vs. ALB

First of all, VPC Lattice is meant to satisfy application layer load-balancing with weighted targets and blue/green (B/G) deployment support.
Moreover, let’s have a look at the target group’s target types available in both cases:

As you can see both TGs support pretty much the same target types with only small differences which doesn’t necessarily mean a given option is missing. E.g. even though the Amazon EC2 Auto Scaling is not explicitly mentioned in the Instances section I managed to successfully attach an ASG as per the diagram above because that option simply exists:

Yes, that same ASG can be attached at the same time to an ALB and a VPC Lattice service. Nothing surprising here as a given service may need to be accessible not only privately to another service but also to clients over a public network.

Moreover, the VPC Lattice service routing configuration consists of listeners and rules just like in the case of ALB, however, all traffic can only be forwarded with no manipulation whatsoever.

Finally, the costs of just running a VPC Lattice service vs. an ALB are almost identical. For example, in the Ireland (eu-west-1) region, it’s $0.0275 vs. $0.0252 per hour.

EKS integration

I believe the Gateway API and Amazon EKS form a more specific use case that deserves a separate touch, therefore it’s not in the scope of this study.
For those interested though, I can tell that there is an AWS Gateway API controller that is meant to let you connect services across different EKS clusters by leveraging Amazon VPC Lattice, and more info on how to do that can be found here.

Admin vs. Developer

A key characteristic when announcing VPC Lattice was that it finally enables the separation of duties between network administrators and developers who can now freely define and manage services themselves.
This reminds me of times shortly after the AWS Lambda introduction and foreshadowing of the no-Ops era. Time will tell also in this case whether no admins involvement in configuring VPC Lattice services is doable or not.
Either way, the idea is that developers simply get a VPC to develop and define their services so then they can share them with admins who control VPC Lattice service networks by:

• associating services and VPCs to networks based on requirements,
• sharing service networks with AWS accounts or organizations,
• enforcing authentication on service network access.

Sharing

Both VPC Lattice services and service networks can be shared with the use of AWS Resource Access Manager (RAM). While services are shared to be associated with different service networks, service networks are shared so that other principals can associate their VPCs and communicate with services associated with those networks.
To better understand what can and cannot be done as a shared resource owner and/or consumer scan the “Responsibilities and permissions for shared resources” section in docs as the information contained there can become very helpful when designing more complex, enterprise-grade architectures and strategies.

Security

The rules defining allowed network communication between individual services living in VPCs are applied with the use of security groups (SGs) configured for every VPC to service network association. VPC Lattice IPv4 and IPv6 managed prefix lists on the other hand simplify the other part of that set-up that involves clients and targets security rules.

Another element of the wider security are auth policies that can be applied on both the service network and service level. They are represented by IAM policy documents and are meant to control in a more granular way what principal has access to which service or a group of services.
Both SGs and auth policies are optional but recommended.

Logging and monitoring

While all out-of-the-box features are nicely described in the docs, one thing worth emphasizing is that both VPC Lattice service and service network logs can be streamed concurrently to:

• CloudWatch Log Group
• S3 bucket
• Kinesis Data Firehose delivery stream

Awesome! Would love to see that for any AWS service, including the ALB.

VPC Lattice wrap-up

Caveats

As Amazon VPC Lattice service is still new, there are some caveats and limitations one should know about. One of them claimed as temporary is the ability to create only an exact match path condition (case insensitive) listener rules in the console. To configure the HTTP match condition for my PoC, I had to use the following AWS CLI command:

$ aws vpc-lattice create-rule \
--name lambda-hh \
--service-identifier svc-0ce77bf32833f5b5b \
--listener-identifier listener-0f287b2d41f2cb905 \
--action '{ "forward": { "targetGroups": [ { "targetGroupIdentifier": "tg-0096d77adfb1bcd29", "weight": 1 } ] } }' \
--match '{ "httpMatch": { "headerMatches": [ { "caseSensitive": false, "match": { "contains": "lambda-hh" }, "name": "My-Header" } ] } }' \
--priority 20

BTW, don’t think you can match the Host header with that rule — it’s not like an ALB with host-based routing.
However, that rule creation through the API resulted in the associated listener becoming uneditable from the console anymore (see screenshot above, the “Edit listener” button is greyed out).

Just because of the abilities delivered with VPC Lattice your appetite for more complex and sophisticated scenarios may grow so don’t forget every VPC can be associated with a single service network at the same time! Therefore, it’s important to design your architecture accordingly.

When working with Lambda functions as targets, as long as your function doesn’t require access to your custom VPC resources there’s no need to configure it with a VPC. Even when you send a request via a VPC Lattice service to a Lambda function set up with a VPC it won’t use the associated ENI in that VPC for communication. Instead, it will always communicate with your function via the Lambda API in the region where it is located, and what is clearly visible above on that CW Logs Insights query output screenshot — see entries without the destinationVpcId.
Need more details on that? Have a look at my “Lambda security paradox” story from 2019.

There is also one thing that I found very easy to miss. Namely, even though the docs say you should have rules allowing traffic from clients to VPC Lattice one may forget to whitelist local clients living in the VPC associated with the service network while only allowing those on the other side of the network. In other words, when configuring a client (like that test-svc-curler Lambda function in the VPC .103) you must:

• [Lambda SG] allow outbound traffic to the VPC Lattice prefix list
• [VPC SG] allow inbound traffic from the Lambda SG

Last but not least, make sure you realize what the default service quotas are and which ones can be adjusted upon request.

Expectations

After watching the re:Invent introductory video I expected a bit more when it comes to path-based routing. Namely, I thought it would allow for defining the entire application routing at the VPC Lattice services level regardless of the target type. Having forwarding as the only available action, when using EC2 or ECS as targets, that path is always forwarded to the backend that must consider that information and know how to deal with such requests and not throw a 404.
While that is not a problem when using Lambda and Gateway API for EKS, I thought it would be great to avoid having to keep routes mapping in sync between infrastructure and application code, especially when they may be kept in different repositories and have independent deployment pipelines.

And hey, where’s private Amazon API Gateway target support!?

The good, the great, and the awesome!

Either way, I have no doubt that Amazon VPC Lattice is a superb improvement over what had to be put in place to satisfy similar private, cross-account, cross-VPC service-to-service comms. While there’s always room for enhancement and more features, that I’m sure AWS will be introducing over time and based on customers' feedback, it has already made things easier by:

• simplifying service-to-service cross-VPC comms,
• mitigating the IP overlap issue,
• enhancing service-to-service comms security,
• facilitating B/G deployments or A/B testing,
• supporting migration and modernization activities,
• and more.

Finally, a great thing about VPC Lattice is that services can be associated with many service networks at the same time providing maximum flexibility and extensibility. Can’t wait to start using it on future projects!