For years, virtualization came with a hidden cost: the ‘Hypervisor Tax.’ Every time your application sent a network packet, your CPU had to stop what it was doing, context-switch, and play the role of a traffic cop. It was a bottleneck that stood between your code and the raw power of the hardware.
The Problem: The “Virtualization Tax”
Before Nitro, AWS used a traditional hypervisor (specifically a customised version of Xen ). In this setup, every time a virtual machine (VM) wanted to send a packet of data, it had to go through a “middleman.”
- Dom0 (The Privileged VM): The physical server ran a specialized VM called “Domain 0” or Dom0. This was a full-blown Linux environment that had direct access to the physical hardware (NICs, Disks).
- Context Switching: When a customer VM (DomU) sent a network packet, the CPU had to “stop” what the customer was doing and “switch” to the Dom0 code to process that packet. This constant back-and-forth is known as a context switch.
- Resource Stealing: Dom0 required its own CPU cores and memory to function. On a large server, you might lose 10% to 20% of the physical hardware capacity just to run the management software. This is the Virtualization Tax.
Overview of how Hypervisor talks to CPU for processing network packets
- Jitter: Because the host CPU is busy “managing” the network for 50 different VMs, latency becomes inconsistent. A packet might be delayed because the CPU was busy processing a storage request for a different customer.
The Solution: AWS Nitro Offloading
The Nitro System effectively “breaks apart” the hypervisor. It takes all the work that Dom0 used to do — networking, storage, and security — and moves it onto a separate piece of hardware: The Nitro Card.
- Hardware Separation: The Nitro Card is a physical PCIe card with its own processor (ASIC) and memory. It is physically separate from the main motherboard where the customer’s CPU (Intel/AMD/Graviton) sits.
Credits: https://dev.to/choonho/nitro-card-why-aws-is-best-46ph
- Zero-CPU Networking: When a VM sends a packet, it goes directly to the Nitro Card via SR-IOV (Single Root I/O Virtualization). The host CPU never has to “touch” the packet. It simply drops it into a memory queue, and the Nitro Card picks it up.
The Nitro Card for VPC
These are independent System-on-Chips (SoCs) connected via the PCIe bus. They run their own operating systems and are responsible for specific tasks like networking, storage, and management.
SR-IOV Implementation
It uses Single Root I/O Virtualization (SR-IOV) to create “Virtual Functions” (VFs). This allows multiple VMs on the same host to have direct, high-speed paths to the hardware without going through a software switch.
The Magic: Direct Memory Access (DMA)
There is a physical controller (the DMA Controller ) on the Nitro Card. This circuit has the electrical authority to take control of the PCIe bus and move data from the system’s RAM directly into the card’s own memory.
Hardware diagram for DMA controller inside NIC
1. Step One: The OS Kernel Writes to RAM (CPU Action)
When an application inside your VM wants to send a packet (e.g., a “Hello World” message), it doesn’t know about hardware. It just hands the message to the OS Kernel. The CPU takes that message and wraps it in standard network headers (TCP, IP, Ethernet). The CPU writes this completed packet into a specific “buffer” in the System RAM (the slice of memory assigned to your VM).
2. Step Two: The “Doorbell” (The Hand-off Signal)
Once the packet is sitting in the RAM, the CPU needs to tell the Nitro Card, “Hey, I’ve left a package for you in the lobby.”
MMIO Write: The CPU writes a tiny bit of data (a “doorbell” signal) directly to a specific address that is physically mapped to the Nitro Card over the PCIe bus.
3. Step Three: DMA Hardware Takeover (No CPU)
The Fetch: The DMA Controller inside the Nitro Card reads the memory address provided by the CPU and reaches across the PCIe Bus.
Direct Copy: It copies the packet data from the System RAM directly into the Nitro Card’s local memory.
The core innovation of the AWS Nitro System is the physical decoupling of the networking data plane from the host CPU. By shifting virtualization tasks to custom hardware, AWS eliminates the “virtualization tax” and provides performance that is nearly indistinguishable from bare metal.
Top comments (0)