Reaching a Model Provider From a Private Subnet With No Internet Gateway

#ai #machinelearning #llm #tutorial

There is no attribute on a subnet called “private”. A subnet is private because of what is missing from its route table, and understanding it that way is what lets you tell in thirty seconds whether your model call is failing because of routing, DNS or a security group — three problems with three unrelated fixes and one identical-looking symptom.

What makes a subnet private

Every subnet is associated with exactly one route table. That table always contains a local route for the VPC’s own CIDR, which is why instances in a VPC can always reach each other. A subnet is public when its route table also has a 0.0.0.0/0 entry whose target is an internet gateway. A subnet is private when it does not. That is the entire distinction.

aws ec2 describe-route-tables \
  --filters Name=association.subnet-id,Values=subnet-0a1b2c3d \
  --query 'RouteTables[].Routes[].[DestinationCidrBlock,GatewayId,NatGatewayId,State]' \
  --output table

If the only rows are the local route and nothing with a 0.0.0.0/0 destination, packets bound for api.anthropic.com have nowhere to go and are dropped inside the VPC. Note the State column: a route can exist and be blackhole, which happens when the NAT gateway or gateway it pointed at has been deleted. A blackhole route looks like a configuration that is present and behaves like one that is absent.

There is a second, subtler reason a public IP will not save you. Attaching a public IP to a task in a subnet with no internet gateway route achieves nothing, because the public IP is translated by the internet gateway — with no route to one, the address is inert. This is why assignPublicIp: ENABLED on an ECS task in a private subnet does not fix outbound and instead usually stalls the task at the image pull.

The symptom is a hang, not an error

A missing route produces silence. The packet is dropped, no ICMP unreachable comes back, and your client sits until its connect timeout expires. That is what you see:

# botocore
botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL:
"https://bedrock-runtime.us-east-1.amazonaws.com/model/.../converse"

# node
Error: connect ETIMEDOUT 160.79.104.10:443

# curl
curl: (28) Failed to connect to api.anthropic.com port 443 after 130002 ms

Compare that to the two neighbours it gets confused with. If DNS is the problem you get a resolution error rather than a timeout — Could not resolve host, getaddrinfo ENOTFOUND, EAI_AGAIN — and the fix is somewhere else entirely, covered in DNS resolution failing for a model endpoint inside a VPC. If a security group is the problem you also get a timeout, which is why you check the route table first: it is the cheaper of the two to rule out. Security group egress on 443 is covered in outbound HTTPS to a model provider.

A useful discriminator: resolve the hostname and then try a raw TCP connect to port 443. If resolution returns an address and the connect hangs, it is routing or filtering. If resolution itself fails, stop — nothing about a NAT gateway will help.

The three exits, and which one you need

AWS gives a private subnet three ways to originate outbound traffic, and they are not interchangeable.

An interface VPC endpoint, over PrivateLink. The traffic never touches the public internet at all. This is the right answer when the model service is an AWS service — Amazon Bedrock publishes endpoint services including com.amazonaws.region.bedrock-runtime. It is not available for a third-party API unless that vendor has published a PrivateLink service. See PrivateLink to a model endpoint.
A NAT gateway. The general answer for IPv4 traffic to anything on the public internet. Your workload keeps no public address of its own; the gateway’s Elastic IP is what the provider sees, which is also what makes it the address you give them for an allowlist. Provisioning and its operational edges are in setting up a NAT gateway for outbound calls.
An egress-only internet gateway. IPv6 only, and free of charge. It permits outbound IPv6 and prevents anything on the internet initiating a connection inbound. Only useful if the provider you are calling has AAAA records — an egress-only internet gateway for a model-calling service covers the case where it does not.

The decision is usually made for you by what you are calling. If it is Bedrock or SageMaker, use the interface endpoint: it is cheaper per GB than NAT, removes an availability dependency, and keeps the traffic inside AWS’s network. If it is a third-party HTTPS API over IPv4, you need a NAT gateway and there is no way around it short of running your own proxy on an instance, which is the same thing with more failure modes.

Two mistakes that look like success

The NAT gateway in the private subnet. A public NAT gateway has to live in a subnet whose route table points 0.0.0.0/0 at an internet gateway — that is, a public subnet. Put it in the private subnet it is meant to serve and it will still create, still report status Available, and still be selectable as a route target. Nothing errors. Traffic simply arrives at the gateway, gets translated, is routed by the same table back to the same gateway, and dies. Two subnets are involved in a working setup and collapsing them into one is silent.

The one route table serving both subnets. If the private subnet is associated with the main route table and you edit the main route table, you may also have edited the public subnet’s routing. Check the associations rather than assuming the names mean what they say — a subnet named private-1a is only private if its route table says so.

A third thing worth planning for rather than calling a mistake: a single NAT gateway lives in one Availability Zone. Workloads in other AZs routing through it pay a cross-AZ transfer charge and lose outbound entirely if that AZ has an event. One NAT gateway per AZ, with each private subnet’s route table pointing at the one in its own zone, is the standard shape and the reason it exists is availability rather than cost.

Proving the path before you deploy

VPC Reachability Analyzer answers the routing question statically, without deploying anything into the subnet, and it is much faster than launching a test instance:

Create a path from the workload’s network interface to the internet gateway on TCP 443 with aws ec2 create-network-insights-path.
Run it with aws ec2 start-network-insights-analysis, then read the result with describe-network-insights-analyses.
Read NetworkPathFound and, when it is false, the ExplanationCode — it names the specific component that dropped the path, such as a missing route or a security group rule, rather than making you infer it.
Only once the path is found, test the application layer: from a task in the subnet, curl -sv https://api.anthropic.com/v1/messages and confirm you get an HTTP status back. A 401 is a complete success at this stage — the packet reached the provider and came home.

Getting an authentication error is the goal. It means routing, DNS, security groups and TLS all worked, and the only thing left is a header — which is a problem you can fix without touching the network.