If you've worked with AWS for more than a few weeks, you've created a VPC. You've drawn subnets, attached route tables, written security group rules, maybe even set up peering between two VPCs. It works. You move on.
But underneath the console clicks and Terraform resources, AWS is orchestrating real network primitives: isolating boundaries, packet filtering, routing decisions, and it is doing this at massive scale. Most of us never see that layer. It's abstracted away, and that's the whole point of a managed service.
I wanted to see it anyway. So I built vpcctl, a Python CLI that replicates core AWS VPC networking behavior: subnets, routing, security groups, peering, using nothing but native Linux primitives: network namespaces, veth pairs, Linux bridges, and iptables. This post walks through what AWS VPC actually does, and how rebuilding the equivalent behavior by hand changed how I think about designing and debugging real AWS networks.
The AWS Abstraction
Before diving into the internals, it's worth being precise about what a VPC actually consists of, since these are the pieces the rest of this post maps against:
- VPC: an isolated virtual network with its own private IP address range (CIDR block).
- Subnet: A subdivision of that CIDR, each tied to an Availability Zone, marked public or private based on its route table configuration.
- Route tables: The rules that decide where traffic goes, either to the internet via an Internet Gateway, or to another subnet, or to a peered VPC.
- Security group: This is the security mechanism on instances. It is a stateful, instance-level firewall that allows or denies traffic based on rules
- VPC peering: A private routing connection between two otherwise isolated VPCs
Every one of these is a logical construct. AWS doesn't have a magic "VPC device" somewhere underneath; it's still doing packet forwarding, filtering, and NAT, just like any other network. The question I set out to answer was: "What does that look like if you build it yourself?"
The Experiment
vpcctl simulates each of these constructs using Linux tools that already exist on any machine:
- Network namespaces stand in for isolated subnets; each namespace has its own routing table and interfaces, completely separate from the host.
- Linux bridges act as the shared L2 segment a subnet's instances sit on
- veth pairs are like physical wires that connect namespaces to bridges, the way an ENI connects an instance to a subnet
- iptables rules enforce security-group-style allow/deny behavior
- NAT via iptables MASQUERADE provides the "public subnet" internet access pattern
The CLI supports creating VPCs, adding subnets (public or private), attaching security group rules, setting up peering between two VPCs, and persisting that state across runs. This is the same operations you'd do in the AWS console, just executed as real Linux networking commands instead of API calls.
Mapping AWS Concepts to Linux Primitives
| AWS Concept | Linux Equivalent | What It's Really Doing |
|---|---|---|
| Subnet isolation | Network namespace | Separate routing table and interface stack; the same reason instances in different subnets can't see each other's L2 broadcast domain |
| Public/private subnet | Route table + NAT (iptables MASQUERADE) | A "public" subnet isn't special; it just has a default route to something with internet access, same as a namespace routed through a MASQUERADE rule |
| Security groups | iptables rules on the veth/bridge interface | Stateful filtering happens at the interface level, not "on the instance," which is exactly why security group changes take effect without a restart |
| VPC peering | Routes between namespaces via a shared bridge or veth link | Peering is just a route. No transit, no NAT, which is also why peered VPCs can't have overlapping CIDRs |
| Route tables | Per-namespace routing tables | Each subnet's "personality" (public vs private) is entirely determined by what's in this table, nothing else |
Seeing these side by side made a few AWS behaviors click that I'd previously just memorized as rules:
Why you can't have transitive peering? where A connects to B and B connects to C, and A is expected to communicate to C through B? This is because each peering connection is just a single static route on each side, added directly to each VPC's route table, pointing straight at the peer. There's no protocol exchanging reachability information between B and A, or between B and C. B has a route to A and a route to C, but nothing tells A "hey, go through B to reach C" because there's no routing intelligence running in between, only two independent point-to-point entries.
Why security group changes are immediate but NACL changes can behave differently? Security groups are stateful: they track connections, so if you allow inbound traffic on a port, the return traffic is automatically allowed out, no matching outbound rule needed. NACLs are stateless: they evaluate every packet independently against explicit rules in both directions, with no memory of the connection. That's the same distinction between an iptables conntrack-based rule (which tracks connection state) and a plain stateless packet filter. It's why NACL misconfigurations are a more common source of "one direction works, the other doesn't" bugs than security group misconfigurations.
Why does subnet CIDR planning matter so much upfront? A network namespace doesn't renegotiate its address space once it's created; the range you assign it is fixed unless you tear it down and rebuild it. AWS subnets work the same way: you can't resize a subnet's CIDR block after creation. If you undersize a subnet early on, the fix later is migrating workloads to a new subnet, not adjusting the existing one. This is why real AWS architectures reserve more IP space up front than they think they'll need.
Why a route table, not a checkbox, is what makes a subnet "public."? There's no attribute on a subnet that says public: true. A subnet is public purely because its route table has a route sending 0.0.0.0/0 traffic to an Internet Gateway. Attach that same Internet Gateway route to a different subnet, and it becomes public too, nothing else changes. This maps directly to how vpcctl decides a namespace's behavior: it's entirely a function of what's in that namespace's routing table, not a flag set anywhere else.
Why peered VPCs can't have overlapping CIDR ranges? Peering just adds a route pointing at the peer's CIDR block. If two VPCs share overlapping address space, that route becomes ambiguous; the system can't tell which destination a given IP actually belongs to. This is the same reason you can't route between two Linux namespaces with identical subnet ranges: routing depends on non-overlapping destinations to make an unambiguous decision.
Why an Internet Gateway attached to a VPC doesn't automatically give every subnet internet access? Attaching the gateway just makes it available as a possible next hop for the VPC. Each subnet still needs its own route table entry pointing to it. This two step design (attach the gateway, then explicitly route to it per subnet) is why you can have a VPC with internet connectivity available but individual subnets still fully private, by simply not adding that route.
What Broke, and What It Taught Me
Building this surfaced a handful of real bugs, each of which mapped back to an "aha" about AWS:
Interface name length limits. Linux caps network interface names at 15 characters (
IFNAMSIZin the kernel). I hit this when generating veth pair names programmatically from VPC and subnet identifiers, since concatenating a VPC ID with a subnet ID blew past that limit, and the interface creation silently failed or was truncated in unexpected ways. This explains something about AWS I'd never questioned: why AWS enforces its own opaque, fixed-format resource IDs (vpc-0a52d63562a6bb5c6,subnet-xxxx) instead of letting you name resources freely and using those names at the networking layer. AWS isn't just being tidy; it's very likely working around the same low-level constraints, just hidden behind an abstraction layer you never see.Unguarded top-level function calls executing at import time. This was a Python structural bug: code that should only run when explicitly called was instead executing the moment the module was imported, because it sat at the top level of the file instead of inside a function or behind an
if __name__ == "__main__"guard. The practical effect was things happening before their dependencies existed. It forced me to think seriously about initialization order, which is exactly the same category of problem that shows up in Terraform when a route resource gets created before the NAT Gateway it depends on exists. Terraform normally infers this dependency graph from resource references, but when that inference fails (implicit dependencies it can't see, or resources defined in a way that hides the relationship), you get the same "this was built before its prerequisite existed" failure, just at the infrastructure layer instead of the code layer.Missing
-j ACCEPTtargets in iptables chains. iptables rules that don't end in an explicitACCEPT(or another terminating target) don't do anything on their own; they just fall through to the next rule, and eventually to the chain's default policy. I had rules that looked correct on paper but never actually accepted traffic because I'd forgotten the terminating target, so packets were silently dropped with zero error output. That's precisely the debugging experience of a "connection timed out" against a misconfigured AWS security group: there's no rejection, no error message, just silence, because a stateful firewall that doesn't explicitly allow traffic doesn't tell you it denied it either. Rebuilding this by hand made that failure mode intuitive rather than mysterious, since I'd actually watched packets vanish for the exact same structural reason.delete_vpcname collisions. My state management identified VPCs by name, and I hadn't accounted for two VPCs being given the same name, which meant a delete operation could match and remove the wrong resource, or fail ambiguously. This is a small-scale version of why AWS enforces strict uniqueness constraints on resource identifiers within an account and region: without a uniqueness guarantee, any operation that looks up a resource by a human-readable identifier becomes unreliable the moment there's a collision. AWS solves this by generating unique IDs regardless of what name tag you attach, so the name is cosmetic and the ID is what every underlying operation actually references.Race conditions between namespace creation and veth attachment. Bringing up a namespace and immediately attaching a veth pair to it sometimes failed if the namespace wasn't fully initialized yet, since these are asynchronous operations at the OS level even though they look sequential in a script. This maps directly onto a very common AWS/Terraform pattern: attaching an ENI, an EBS volume, or a security group to an EC2 instance immediately after the instance is launched, before it's actually in a running state. Both cases are the same underlying lesson: that "created" and "ready to be attached to" are not the same moment, and code that assumes otherwise will fail intermittently rather than consistently, which makes it worse to debug.
Takeaway
None of this changes how you'd actually configure a VPC in the console or in Terraform. But it changes how you debug one. When a security group rule "isn't working," or a peering connection isn't routing traffic, or a private subnet can't reach the internet, knowing that these are, underneath, just routing tables and stateful packet filters makes the failure modes obvious instead of opaque.
If you want to build the same intuition, the fastest path isn't reading more VPC documentation; it's picking one AWS networking concept and asking "what would I have to build, from raw Linux primitives, to reproduce this behavior?" You'll hit the edge cases AWS handles for you, and you'll understand exactly why they exist.
vpcctl is written in Python and available on GitHub.
Top comments (1)
Nicely written!