Building Secure AWS Cloud Networking: VPC Peering, Private EC2, NAT Gateway, and S3 Gateway Endpoint

A Hands-On Implementation of Multi-VPC Connectivity, Controlled Internet Access, IAM-Based Authentication, and Private AWS Service Routing

By Syed Bilal Bukhari
Cloud Computing / AWS / Cloud Networking · Foundational to Intermediate · Region: US East (N. Virginia), us-east-1

---

Introduction

In cloud networking, "it connects" and "it connects the way I intended" are different statements. A packet leaving an EC2 instance is steered by route tables, filtered by security groups, translated by gateways, and authorized (for AWS APIs) by IAM. When something fails, the cause sits in one of those layers, and you can only find it if you know which path the traffic is supposed to take.

I built this lab to make those paths explicit. It has two VPCs in us-east-1, one public EC2 instance, and one private EC2 instance with no public IPv4 address. The private instance has three jobs: talk to the public VPC over VPC Peering, reach the internet outbound-only through a NAT Gateway, and reach Amazon S3 using an IAM role instead of stored access keys.

This is a foundational-to-intermediate lab, not a production design. I will point out where it falls short as I go. I will also separate what the AWS Console configuration shows from what the terminal tests prove, because those are two different kinds of evidence.

Project Objectives

1. Give the public EC2 instance internet access through an Internet Gateway.
2. Connect two VPCs privately using VPC Peering, with non-overlapping CIDR blocks.
3. Reach the private EC2 instance over SSH through the public instance as a jump host.
4. Let the private instance start outbound internet connections through a NAT Gateway, without a public IP.
5. Add an S3 Gateway Endpoint so S3-bound traffic from the private route table has a dedicated route.
6. Authenticate the private instance to AWS with an IAM role rather than permanent access keys.
7. Validate each step with real terminal tests and Console configuration.

AWS Services and Tools

Component	What it did in this lab
Amazon VPC (two VPCs)	Isolated networks: mys-vpc-001 and myp-vpc-002
Subnets and route tables	Decide what is "public" or "private" through their routes
Internet Gateways	One per VPC; the target for internet-routable traffic
VPC Peering	Private routing between the two VPCs
NAT Gateway + Elastic IP	Outbound-only IPv4 internet access for the private subnet
S3 Gateway VPC Endpoint	Route-table target for regional S3 traffic
IAM role	Temporary credentials for the private instance
Security Groups	Instance-level traffic filtering (ICMP, SSH)
EC2 (t3.micro, Amazon Linux 2023)	One instance in each VPC
MobaXterm, ping, curl, AWS CLI	SSH access and connectivity verification

Architecture Overview

Figure 1. Architecture of the lab. Orange arrows are internet-bound traffic, the dashed blue line is inter-VPC traffic over peering, and green arrows are S3 traffic through the Gateway Endpoint. The private EC2 instance has no direct path to an Internet Gateway; its internet traffic goes through the NAT Gateway first.

	Public VPC	Private VPC
Name / CIDR	mys-vpc-001 / 10.0.0.0/24	myp-vpc-002 / 10.0.1.0/24
Subnets	mys-subnet-01 10.0.0.0/28 (us-east-1a)	myp-private-subnet-002 10.0.1.0/28 and myp-nat-public-subnet 10.0.1.16/28 (both us-east-1b)
EC2	my-public-ec2, 10.0.0.6, public IPv4	Private EC2, 10.0.1.6, no public IPv4
Gateways	Internet Gateway	myp-igw-002, NAT Gateway myp-nat-gateway-002 (10.0.1.22 + Elastic IP)

Public VPC Configuration

The Public VPC has one public subnet. The instance my-public-ec2 is a t3.micro running Amazon Linux 2023, with private IP 10.0.0.6 and a public IPv4 address. I reached it over SSH with key authentication from MobaXterm. It plays two roles: the internet-facing test host and the jump host for the private instance.

Figure 2. The EC2 console showing running t3.micro instances that passed 3/3 status checks. my-public-ec2 is in us-east-1a; the instance used for private-side testing is in us-east-1b.

A subnet is public because of its route table, not its name. mys-public-rt sends VPC-local traffic locally, sends 0.0.0.0/0 to the Internet Gateway, and sends 10.0.1.0/24 to the peering connection.

Figure 3. mys-public-rt (captured from the Edit routes view). All three routes are Active: local, 10.0.1.0/24 via the peering connection, and 0.0.0.0/0 via the Internet Gateway.

Private VPC Configuration

The VPC is called "Private VPC", but that name is only a label. Whether a subnet is public or private depends on its route table. This VPC deliberately contains one of each:

• myp-private-subnet-002 (10.0.1.0/28) holds the private EC2 instance at 10.0.1.6. Its route table has no route to an Internet Gateway.
• myp-nat-public-subnet (10.0.1.16/28) holds only the NAT Gateway. Its route table, myp-nat-public-rt, has a default route to the Internet Gateway myp-igw-002, which makes it a public subnet.

Figure 4. The Private VPC resource map: two subnets in us-east-1b, three route tables, and the connections myp-igw-002, myp-nat-gateway-002 (public NAT, one ENI and one EIP) and myp-s3-endpoint (gateway endpoint to S3). The right edge of the original capture is cut off.

VPC Peering

Concepts

VPCs are isolated by default. A VPC Peering connection creates a private routing path between two VPCs using private IP addresses. It is not a gateway and not a VPN, and it does not provide internet access.

Four properties shaped the design:

• Non-overlapping CIDRs are mandatory. If both VPCs used the same range, a route for that range could not say which VPC a destination belongs to. Here the VPCs use 10.0.0.0/24 and 10.0.1.0/24.
• Request and accept. One VPC (the requester, mys-vpc-001) asks and the other (the accepter, myp-vpc-002) accepts. Until it is accepted, the connection is not usable.
• Both sides need routes. An Active peering connection only creates the possibility of traffic. Each VPC's route table needs a route to the other VPC's CIDR with the peering connection as the target.
• Peering is not transitive. AWS documents that two VPCs cannot use a third, peered VPC as a transit path. Peering also does not bypass Security Groups or Network ACLs, which still filter the traffic.

Figure 5. public-private-peering (pcx-0763ca5cff1e5a68e) in Active status, with requester VPC vpc-044079de0026185ef. The accepter column is off-screen in this capture, so the accepter details come from my project notes.

Routes on both sides

The Public VPC's route (Figure 3) sends 10.0.1.0/24 to the peering connection. The Private VPC's route table, myp-private-rt-002, has the mirror route plus two more:

Destination	Target	Purpose
10.0.1.0/24	local	Traffic inside the Private VPC
10.0.0.0/24	Peering connection	Traffic to the Public VPC
0.0.0.0/0	NAT Gateway nat-04d646b2df4be7880	Everything else, IPv4
pl-63a5400a	Gateway endpoint vpce-0f6dfcad71b59e9d9	S3 (regional prefix list)

Figure 6. myp-private-rt-002 with all four routes Active. This is also the evidence that the S3 prefix-list route exists (see the S3 section).

AWS picks the most specific matching route (longest prefix match). A packet to 10.0.0.6 matches 10.0.0.0/24 and goes over peering, not to the NAT Gateway, even though 0.0.0.0/0 also matches. A packet to an S3 address falls in a range from the S3 prefix list, which is more specific than 0.0.0.0/0, so it takes the endpoint route. Anything else falls through to the NAT Gateway.

Reachability test

From the public instance (10.0.0.6) I pinged the private instance:

ping -c 4 10.0.1.6

Figure 7. Four packets transmitted, four received, 0% loss. Replies took between 0.452 and 0.468 ms. The final min/avg/max line is cut off in my capture; the mean of the four replies is about 0.457 ms.

This proves ICMP reachability between instances in separate VPCs. It does not prove that every application port works, because ping only exercises ICMP. The first attempts were blocked until I added the right Security Group rule (see Challenges).

SSH Access Between VPCs

I confirmed that the public instance could reach port 22 on the private instance over the peering link. For access from my own machine I used MobaXterm's SSH gateway (jump host) feature, which opens the session to the private address through the public instance, so the private machine never needs a public IPv4 address. I did not copy the private key onto the jump host.

Port 22 on the private instance should accept connections only from the jump host's side, not from 0.0.0.0/0. I did not capture screenshots of the Security Group rules, so I am describing that intent rather than showing the rule set.

NAT Gateway for Outbound Internet Access

The private instance has no public IP, so by itself it cannot reach the internet. A public NAT Gateway in a public subnet fixes that:

Private EC2 → private route table → NAT Gateway → Internet Gateway → Internet

Why each piece is there:

• The NAT Gateway is in a public subnet (myp-nat-public-subnet) because it needs a route to an Internet Gateway, and it has an Elastic IP that serves as its public address.
• The NAT subnet's route table points to myp-igw-002. Without that, the NAT Gateway has no way out.
• The private instance keeps only its private IP. The NAT Gateway translates its source address on the way out and maps replies back.
• Inbound is limited by design. AWS documents that connections through a NAT Gateway must start from inside the VPC, so unsolicited inbound connections from the internet cannot reach the private instance this way.

A NAT Gateway does not replace Security Groups, and it is not application-level security. It simply provides address translation for outbound IPv4.

Figure 8. myp-nat-gateway-002 in Available state: connectivity type Public, private IPv4 10.0.1.22, in myp-nat-public-subnet of myp-vpc-002. The account ID in the ARN and the Elastic IP are redacted.

From the private instance:

curl -I --max-time 15 https://www.google.com

Figure 9. HTTP/2 200 received from the private EC2 prompt (ip-10-0-1-6). I cropped the screenshot to remove the cookie headers that followed.

Before the routing was right, this same command timed out. That is covered in Challenges.

S3 Gateway Endpoint and IAM Role

The endpoint

A VPC endpoint lets resources in a VPC reach an AWS service without going over the public internet path. There are two types: Interface endpoints (an ENI with private IPs in your subnet, powered by PrivateLink) and Gateway endpoints (a route-table target, available for S3 and DynamoDB). I used a Gateway endpoint because it fits the lab: it needs no ENI in my subnet, and AWS states there is no additional charge for it.

Because it works through routing, it is associated with route tables. When I associated myp-private-rt-002, AWS added the route pl-63a5400a → vpce-0f6dfcad71b59e9d9, where pl-63a5400a is the managed prefix list for regional S3 addresses. That route is visible in Figure 6.

Figure 10. myp-s3-endpoint (vpce-0f6dfcad71b59e9d9): type Gateway, service com.amazonaws.us-east-1.s3, VPC myp-vpc-002, status Available.

Three limits are worth stating. The endpoint covers S3 only, so other traffic still needs the NAT Gateway. It serves only the route tables it is associated with, and AWS documents that resources across a peering connection cannot use it. And an endpoint policy and an IAM policy do different jobs: one limits what can pass through the endpoint, the other limits what the caller may do. My screenshots show no custom endpoint policy, so I am not claiming one was applied.

The IAM role

Initially, aws configure list showed that credentials were not configured. After attaching an IAM role to the private instance, it reported:

access_key : ************ : iam-role
secret_key : ************ : iam-role
region     : us-east-1    : imds

The credentials come from the EC2 Instance Metadata Service as temporary credentials that rotate, so no access key is stored on disk. This is the approach AWS recommends over putting long-term keys on an instance. The network endpoint and IAM are separate: the endpoint provides a route; IAM decides whether the call is allowed.

My screenshots do not show the role's name or policy, so I am not stating what permissions it had. For a production version I would scope the role to the specific bucket and actions, for example s3:ListBucket on one bucket and s3:GetObject on the prefixes actually needed, rather than broad S3 access. Note that aws s3 ls with no arguments also needs s3:ListAllMyBuckets, which a tightly scoped role might deliberately omit.

S3 listing

aws s3 ls s3://supplychain-de-2026-pk84726/

Figure 11. The private instance lists the historical/ and raw/ prefixes of an existing bucket.

I want to be precise about what this shows. The listing proves authenticated S3 API access from the private instance. The route table (Figure 6) shows that the intended network path for S3 addresses is the Gateway Endpoint. But the private subnet also has a NAT default route, so this command would likely have worked through NAT too. I did not run a packet trace or flow-log analysis, so I am not claiming that this specific request used the endpoint. To prove that, I would temporarily remove the NAT default route and confirm the S3 listing still works while curl to Google fails.

Validation and Testing

Test	Command (run from)	Observed result	Evidence
A. Public EC2 internet	curl -I --max-time 15 https://www.google.com (public EC2)	HTTP/2 200	Recorded in my notes; not pictured
B. Peering reachability	ping -c 4 10.0.1.6 (10.0.0.6)	4 sent, 4 received, 0% loss	Figure 7
C. SSH reachability	SSH to port 22 on 10.0.1.6; MobaXterm jump host	Connected	Recorded in my notes; not pictured
D. Private EC2 internet via NAT	curl -I --max-time 15 https://www.google.com (10.0.1.6)	HTTP/2 200	Figure 9
E. AWS CLI present	aws --version	AWS CLI 2.36.47	Recorded in my notes; not pictured
F. IAM role credentials	aws configure list	Credentials from iam-role; region from imds	Recorded in my notes; not pictured
G. S3 service access	aws s3 ls	Listed an accessible bucket	Recorded in my notes; not pictured
H. S3 prefix listing	aws s3 ls s3://supplychain-de-2026-pk84726/	PRE historical/, PRE raw/	Figure 11

Console figures (3 to 6, 8, 10) show how the network is configured; the terminal tests (B, D, H) show that traffic and API calls worked.

Challenges and Troubleshooting

Overlapping VPC CIDR blocks. My first pair of VPCs had overlapping ranges. VPC Peering requires non-overlapping CIDRs, so that pair could not be peered as designed. I created a replacement VPC, myp-vpc-002, with 10.0.1.0/24, which is the one used throughout this article.

Peering "Pending Acceptance" and a blackhole route. A route that points at a peering connection that has not been accepted shows a blackhole status. The cause was the pending request; accepting it moved the connection to Active, and the routes then showed Active (Figures 3 and 6).

Ping blocked by Security Groups. The route was in place, but ICMP did not get through until I added the appropriate Security Group rule. Routing and filtering are separate layers, and peering does not bypass the filter.

NAT internet access timed out. Before the route-table configuration was corrected, curl from the private instance timed out. Two routes have to be right for this path: the private route table's default route to the NAT Gateway, and the NAT subnet's default route to the Internet Gateway. After correcting the routing, the same command returned HTTP/2 200.

AWS CLI credentials missing. The instance had no role, so the CLI had no credentials. Attaching an IAM role fixed it, and aws configure list then showed iam-role.

S3 endpoint route verification. After creating the endpoint, I checked the private route table and confirmed the pl-63a5400a route was Active rather than assuming the association worked.

Security Considerations

• Subnet isolation. The private instance has no public IPv4 address and no inbound route from the internet. Its only inbound path is over the peering link, including SSH forwarded through the jump host.
• Security Groups remain the instance-level control. Port 22 should be limited to the jump host, and ICMP opened only as needed.
• The jump host concentrates access in one place, but it is itself an exposed system that needs patching and restriction. A managed option such as AWS Systems Manager Session Manager could avoid an open SSH port; I did not implement it.
• IAM roles avoid stored keys but still need least-privilege policies, which I have described rather than demonstrated.
• NAT blocks unsolicited inbound connections but does not filter outbound traffic by destination.
• An S3 endpoint policy could restrict which buckets are reachable; I have not shown one.

The tests show the network behaves as designed on the paths I tested. That does not make the design hardened.

Cost Considerations

A lab like this is cheap only while you remember to remove it.

• NAT Gateway is billed per hour while it exists, plus per GB processed. In a small lab it is usually the largest line item.
• Public IPv4 addresses, including the NAT Gateway's Elastic IP and the public instance's address, are billed.
• EC2 t3.micro instances accrue instance-hours while running.
• Data transfer. AWS documents that peering traffic within one Availability Zone is free. My instances sit in different AZs (us-east-1a and 1b), so inter-AZ charges could apply to larger transfers; my ping and curl tests are trivial. The NAT Gateway is in the same AZ as the private instance, which avoids an extra cross-AZ hop on that path.
• The S3 Gateway Endpoint has no additional hourly or processing charge.

Prices change, so check the VPC pricing page rather than trusting figures in an article. When you finish, terminate the instances, delete the NAT Gateway, and release its Elastic IP, which keeps billing if left allocated.

Key Lessons Learned

1. A route table defines "public" and "private". The Private VPC contains a public subnet because the NAT Gateway needs one.
2. An Active peering connection moves no packets by itself. It needs routes on both sides and permissive Security Groups.
3. A timeout usually points at a missing route. The NAT timeout was a routing problem, not a problem with the destination.
4. Longest-prefix matching lets the routes coexist. The peer CIDR and the S3 prefix list win over the 0.0.0.0/0 default.
5. Configuration evidence and test evidence answer different questions. A successful aws s3 ls does not by itself show which path carried the request.
6. Connectivity and authorization fail separately. The endpoint gave a route; the IAM role gave permission.

Conclusion

The lab verified these outcomes: the public instance reached the internet; two non-overlapping VPCs were joined by an Active peering connection with a 4/4 ping and 0% loss; the private instance, with no public IPv4 address, made outbound HTTPS requests through a NAT Gateway; and it listed an S3 bucket's prefixes using IAM role credentials, with an Active S3 Gateway Endpoint route in its route table.

A production-oriented version would add NAT Gateways per Availability Zone (or the regional NAT option), least-privilege IAM and an S3 endpoint policy, documented Security Group rules, a managed access path instead of an SSH jump host, VPC Flow Logs, and infrastructure-as-code. I also have not shown that S3 requests used the endpoint, which a route-removal test or flow logs could settle.

References

• Amazon VPC documentation: https://docs.aws.amazon.com/vpc/
• VPC peering connections: https://docs.aws.amazon.com/vpc/latest/peering/working-with-vpc-peering.html
• What is VPC peering (Peering Guide): https://docs.aws.amazon.com/AmazonVPC/latest/PeeringGuide/Welcome.html
• VPC peering, connectivity options whitepaper: https://docs.aws.amazon.com/whitepapers/latest/aws-vpc-connectivity-options/vpc-peering.html
• Configure route tables: https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Route_Tables.html
• How route priority works: https://docs.aws.amazon.com/vpc/latest/userguide/route-tables-priority.html
• Internet gateways: https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Internet_Gateway.html
• NAT gateways: https://docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-gateway.html
• Gateway endpoints: https://docs.aws.amazon.com/vpc/latest/privatelink/vpce-gateway.html
• Gateway endpoints for Amazon S3: https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html
• IAM roles for Amazon EC2: https://docs.aws.amazon.com/IAM/latest/UserGuide/roles-usingrole-ec2instance.html
• Amazon VPC pricing: https://aws.amazon.com/vpc/pricing/