How QEMU Emulates Hardware: Device Models, MMIO, and TCG

The previous article established that QEMU emulates the devices KVM deliberately doesn't.

We constantly hear the phrase, “QEMU emulates devices.” It is treated like a magic trick. A user-space program somehow conjures a virtual disk or a network card out of thin air, and a guest operating system is perfectly fooled.

But there is no black box. The entire illusion relies on a single mechanism which is applied consistently across every piece of virtual hardware. Once you see this mechanism, every performance characteristic of virtualization makes sense. Why does VirtIO exist, why certain operations feel catastrophically slower in a VM than on bare metal.

Before We Begin: QEMU is Two Different Programs

We need to clear up a common point of confusion before looking at the code or going further. QEMU actually does two entirely different jobs depending on how you launch it.

Full-system emulation (qemu-system-*) builds a virtual environment from scratch to boot a complete guest operating system. This is what Libvirt launches for every VM, and it is the sole focus of this series. For example, qemu-system-aarch64 constructs a complete, virtualized motherboard from scratch, including the CPU, RAM, disk controllers, network cards, and firmware—allowing a full guest OS to boot.

User-mode emulation (qemu-*) executes a single binary compiled for a foreign architecture directly on your host OS by translating its system calls on the fly. This engine doesn't build a machine or boot an OS. It's how you run an x86 binary on an ARM machine without a VM. And it is what Docker's binfmt_misc cross-architecture support uses under the hood. This is useful, but not virtualization.

Note that, this series is strictly about full-system emulation. If a command begins with qemu-system-, it is fabricating an entire computer.

The trick: devices live at memory addresses

The mechanism that makes emulation possible relies on a fundamental truth about real hardware:

When an operating system wants to send a byte to a serial port, it doesn't call a function. Instead, it writes to a specific memory address. This memory address is physically wired directly to the serial chip on the motherboard. As you can see no RAM is ever involved here. The motherboard's address decoder routes the signal straight to the hardware registers instead.

This is called memory-mapped I/O, or MMIO, and it's how the CPU talks to essentially every device on a modern ARM system. So, the devices are not special entities the CPU has a private channel to. The devices are just regions of the address space.

Which means the CPU's entire interface to a device comes down to this: writing to certain addresses and reading from certain addresses. That's it. Nothing else. And this is an interface software can easily impersonate!

QEMU maps out a guest address space where specific regions are intentionally flagged as "device space" rather than RAM. Now, let's imagine this guest attempts to access one of these coordinates. This operation is blocked from executing as a memory task. Instead, it triggers a trap. Then QEMU runs a dedicated software function.

That software function is the device. For example, a virtual UART is just a few hundred lines of C code that prints a character to your terminal when a write occurs. A virtual disk controller is C code that intercepts a guest read request and fetches bytes from a .qcow2 file on your host SSD.

Let's me show you this in code. The the following is a simplified, structural layout of what that C code of this software function looks like. Don't overwhelm yourself with all the details in the code. Just take a look at it. I just want show you that there is no magic here.

1. The Handoff Interface (MemoryRegionOps)

First of all, QEMU needs to map the memory region to specific C functions. It registers a structure that tells the engine exactly what code to execute when a guest touches the device's addresses:

/* This structure acts as the router for guest hardware accesses */
static const MemoryRegionOps uart_mmio_ops = {
    .read = uart_mmio_read,   /* Function to run when guest reads from an address */
    .write = uart_mmio_write, /* Function to run when guest writes to an address */
    .endianness = DEVICE_NATIVE_ENDIAN,
    .valid.min_access_size = 1,
    .valid.max_access_size = 4,
};

2. The Software Function (uart_mmio_write)

Let's imagine that the guest writes to 0x09000000. When it does, the hardware trap catches it, and QEMU executes this exact type of software function. You can see that it is just a standard C switch statement mapping address offsets to software behaviors.

/* The software function that IS the device hardware channel */
static void uart_mmio_write(void *opaque, hwaddr offset, 
                            uint64_t value, unsigned size)
{
    UARTState *s = (UARTState *)opaque;

    /* The offset is how far past the base address (0x09000000) the guest wrote */
    switch (offset) {
        case 0x00: // Data Register (TX Buffer)
            /* The guest wants to transmit a character. 
               Instead of triggering silicon, we print it to your host terminal. */
            putchar((char)value); 
            fflush(stdout);

            /* Tell our software model that an interrupt is ready (Data sent) */
            s->uart_status |= STATUS_TX_EMPTY; 
            break;

        case 0x04: // Interrupt Enable Register
            /* The guest is trying to toggle hardware interrupt lines */
            s->interrupt_mask = value;
            break;

        case 0x08: // Baud Rate Divisor
            /* A real chip would change electrical frequencies here.
               QEMU just updates an integer variable in RAM. */
            s->baud_rate = 115200 / (value ? value : 1);
            break;

        default:
            /* Guest tried to write to an unmapped register address */
            log_bad_hardware_access(offset);
            break;
    }
}

There is no magic anywhere in this. Every device your guest sees is a software model like this. A chunk of code pretending to be a chip. It does this well by responding to the same address accesses the real chip would respond to. And the guest's driver cannot tell the difference, because from the driver's perspective there is no difference.THe guest driver writes to an address and something happens.

Why the guest's firmware finds these devices at all

This architecture raises an obvious question: how does the guest OS know that a serial port is exactly at 0x09000000 in the first place?

On a physical ARM board, this layout is explicitly dictated by something called a Device Tree. A device tree is a structured data file compiled into the firmware that outlines exactly what hardware components exist and what memory addresses they occupy. x86 has legacy hardware discovery conventions, but ARM has no equivalent of this. I won't go into details about x86 architecture because the series is about ARM. Just remember that in ARM the entire motherboard topology must be spelled out line-by-line for the kernel at boot.

QEMU generates a this device tree describing the machine it's emulating, and hands it to the guest at boot. The guest kernel reads it and sees "there's a PL011 UART at 0x09000000," loads the PL011 driver, and starts writing to that address. The driver is the standard, unmodified Linux driver for real PL011 hardware. It has no idea it's talking to C code.

The device tree

Here is a highly simplified, annotated snippet of a real Device Tree Source (.dts). Again, just take a look only to recognize there is no magic.

dts/dts-v1/;

/ {
    interrupt-parent = <0x8001>;
    #address-cells = <0x02>;
    #size-cells = <0x02>;

    /* Tells the Linux kernel this is QEMU's standardized virtual board */
    model = "linux,dummy-virt";              /* Identifies this as the synthetic 'virt' board */
    compatible = "linux,dummy-virt";

    /* Core CPU Configuration (e.g., 2 vCPUs requested) */
    cpus {
        #address-cells = <0x01>;
        #size-cells = <0x00>;

        cpu@0 {
            device_type = "cpu";
            compatible = "arm,cortex-a57";
            reg = <0x00>;
        };

        cpu@1 {
            device_type = "cpu";
            compatible = "arm,cortex-a57";
            reg = <0x01>;
        };
    };

    /* System Memory Layout (e.g., -m 2048M starts at base address 0x40000000) */
    memory@40000000 {
        device_type = "memory";
        reg = <0x00 0x40000000 0x00 0x80000000>;
    };

    /* The Interrupt Controller (GIC) so the CPU can talk to peripherals */
    intc@8000000 {
        compatible = "arm,gic-v3";
        interrupt-controller;
        #interrupt-cells = <0x03>;
        reg = <0x00 0x08000000 0x00 0x010000>,   /* GIC Distributor physical address */
              <0x00 0x080A0000 0x00 0xF60000>;   /* GIC Redistributors physical address */
        phandle = <0x8001>;
    };

    /* A Virtual UART (Serial Port) for console output */
    pl011@9000000 {
        compatible = "arm,pl011", "arm,primecell"; /* Tells Linux which driver binary to load */
        reg = <0x00 0x09000000 0x00 0x1000>;      /* Base address (0x09000000) and memory size (4KB) */
        interrupts = <0x00 0x01 0x04>;            /* Bound to SPI interrupt 1 */
        clocks = <0x8000>;                         /* Tells the guest which hardware interrupt wire it uses */
        clock-names = "uartclk";
    };

    /* ... PCI host bridge, virtio devices, and flash memory nodes continue below ... */
};

At the very top, QEMU defines the basic structure of our synthetic motherboard. It explicitly states that this is not a real-world chip from Raspberry Pi or Apple, but QEMU’s custom engine.
Take a look at the pl011@9000000 block of the code at the end. This is a Virtual UART.

This is also why the virt machine type matters on ARM64. Because it is not emulating any real-world board. It is a synthetic machine defined by QEMU. On x86, machine types like pc and q35 emulate actual historical chipsets, complete with their quirks. ARM64 skipped that inheritance.

The Dual-Engine Architecture of QEMU

So QEMU has a device job. It also has a CPU job — and how it handles the second is exactly where KVM enters.

QEMU manages peripheral hardware. It has a device job. This should be clear to you by now. But, it also faces a separate challenge: how to execute the guest's CPU instructions. This is QEMU's CPU job.

Half 2 never changes. We discussed in the previous article, that QEMU even without a KVM still able to virtualize. But, it is just slower. The peripheral devices are emulated by QEMU in userspace even if the KVM is missing or inactive. KVM has no built-in device models and never acquires any previous article. (There is a minor exception: KVM implements a few latency-critical components, like the interrupt controller, inside the host kernel to boost performance—but the general rule stands, and that exception is simply an optimization worth remembering.)

Half 1 is the part with two implementations.

TCG — the Tiny Code Generator — This is QEMU's software CPU. It reads blocks of guest instructions, compiles them into equivalent host instructions on the fly, caches the resulting binaries, and runs them. It is a Just-In-Time (JIT) compiler operating between two processor architectures. This is true software emulation. And this is your only option when your guest and host architectures don't match. An x86 guest running on an ARM64 host must use TCG; no amount of hardware assistance can force an ARM core to natively parse x86 machine code.

KVM is the hardware path from article 1.2. Instead of translating instructions in software, QEMU asks KVM to orchestrate the execution. KVM runs the guest code directly on a real, physical core using hardware virtualization extensions. And after that KVM hands control back up to QEMU only when a device trap occurs.

The choice between them is what <domain type='kvm'> versus <domain type='qemu'> selects in libvirt's XML, and it's the single biggest performance factor in the entire stack. We will learn more about these in future posts.

The consequence: what a device access costs

When you combine QEMU's two halves, the central performance reality of all virtualization instantly falls into right place.

Under KVM, guest code runs natively at full speed. Arithmetic, conditional loops, and memory operations inside actual RAM happen directly on the bare metal without any intervention from KVM or QEMU. Millions of instructions can execute back-to-back without either host component ever waking up.

However, a device access can't complete natively, because there's no physical device. It has no choice but to trap out to QEMU's C code.

So the cost model of a virtual machine is: native speed for computation, and a relatively expensive round trip for every device interaction.

This single asymmetry explains an enormous amount of real-world behavior. If you have ever worked with VMs you will be able to relate immediately. It is the exact reason why CPU-bound workloads in a VM perform at near-bare-metal speeds while I/O-heavy workloads lag. It is why a virtual serial console is perfectly fine for login shell. Because a human types slowly. So a trap per character goes unnoticed. However, it would be absolutely catastrophic for a virtual network card processing a hundred thousand packets per second.

And it's the reason VirtIO exists. If device access is the expensive operation, the winning strategy is to make each one carry more work. In other words: you batch. That's article 6.

But you would understand the batching argument only if you know what the round trip actually involves: what "control leaves the guest" means mechanically, what data crosses the boundary, and why some traps are far cheaper than others. That's the next article.

Summary

• qemu-system-* emulates a whole machine and boots an OS. qemu-<arch> runs a single foreign binary. Only the first is virtualization.
• Devices on real hardware are reached through memory-mapped I/O — the CPU writes to addresses that are wired to chips rather than RAM.
• QEMU exploits this by marking regions of the guest address space as device regions. An access there traps and runs a C function. That function is the device.
• On ARM64, the guest discovers what exists and where via a device tree QEMU generates. Guest drivers are the standard, unmodified Linux drivers.
• QEMU has two halves. Devices are always emulated in QEMU userspace. The CPU is either translated in software (TCG) or run natively on real hardware (KVM).
• Under KVM, computation runs at native speed but every device access costs a trap out to QEMU. That asymmetry is the foundation of every performance decision in virtualization.