DEV Community

Cover image for Debugging the Ubuntu 6.8 Kernel in GDB & QEMU — Without Rebuilding It
Harrison Guo
Harrison Guo

Posted on Originally published at harrisonsec.com

Debugging the Ubuntu 6.8 Kernel in GDB & QEMU — Without Rebuilding It

#linux #debugging #c #programming

You can attach GDB to a stock Ubuntu 6.8 kernel running in QEMU and step through it with full symbols — no custom build, no kernel compile. The catch is a quiet one: you'll connect successfully, set a breakpoint on a function that fires constantly, hit continue, and nothing will happen. The kernel is running. GDB is attached. And every breakpoint sits at an address the kernel is no longer using. This is the bug that isn't in any source file, and the fix is one boot parameter.

The setup, no rebuild required

You don't need to compile a kernel to debug one. Ubuntu ships the debug symbols separately:

  • The stock kernel image you already boot.
  • The matching vmlinux with symbols, from the linux-image-*-dbgsym package (Ubuntu's ddeb debug archive). This is the piece most people skip — it gives you the symbol file without rebuilding anything.
  • QEMU hosting the kernel, with the GDB stub open.
  • GDB on the host, pointed at that vmlinux. 
qemu-system-x86_64 -kernel vmlinuz-6.8 -initrd initrd.img \
  -append "console=ttyS0" -s -S -nographic
# -s = gdb stub on :1234, -S = freeze at reset until GDB connects

gdb vmlinux-6.8
(gdb) target remote :1234
(gdb) break schedule
(gdb) continue
Enter fullscreen mode Exit fullscreen mode

schedule runs thousands of times a second. If your breakpoint is real, it fires immediately.

The symptom: a breakpoint that never fires

It doesn't fire. The connection is live, the kernel is clearly running (you can see it boot on the serial console), but GDB sits there. Set a breakpoint anywhere and it's the same — silence. Nothing is wrong with your GDB, your QEMU, or your commands.

Root cause: KASLR moved the kernel, your symbols didn't

Modern kernels ship with KASLR (Kernel Address Space Layout Randomization, CONFIG_RANDOMIZE_BASE). At every boot it slides the kernel's base address by a random offset, so an attacker can't assume where kernel code lives. Your vmlinux symbol file, by contrast, is static — it describes the addresses the kernel would have at its default base. The running kernel is somewhere else entirely, shifted by a random amount chosen this boot.

So when GDB plants a breakpoint on schedule, it writes it at the compile-time address. The CPU never executes there, because the real schedule is at that address plus the KASLR offset. GDB and the kernel are looking at the same function through two address spaces that drifted apart the moment the machine booted. Everything about the breakpoint is correct except the one thing that matters — where it is.

The fix: turn KASLR off at boot

Add one word to the kernel command line:

-append "console=ttyS0 nokaslr"
Enter fullscreen mode Exit fullscreen mode

nokaslr tells the kernel not to randomize its base, so it loads exactly where the static vmlinux says it should. Reconnect, break on schedule, and it fires on the first scheduler tick. No rebuild was ever needed — the symbols were always right; only the alignment was missing.

When disabling KASLR isn't acceptable — you're chasing a bug that only reproduces with it on — you don't rebuild either. You compute the offset. Find the runtime address of a known symbol (the kernel logs a KASLR line early in dmesg, and /proc/kallsyms gives runtime addresses when it isn't restricted), subtract the static address of the same symbol, and that delta is the kaslr_offset. Feed it back to GDB with add-symbol-file vmlinux <.text address + offset> and the whole map shifts into place. Same idea, done by hand.

Why this is a bug class, not a kernel trick

Strip away the kernel specifics and this is a debugger looking in the wrong address space — and that shape shows up far from kernel work:

  • Coredump symbolication. A PIE Go binary loads at a randomized base too. Symbolicate a core with symbols resolved against a different base or a slightly different build and you get stack traces with the right shape and the wrong addresses — plausible, confident, and pointing at a function that wasn't running. Hours vanish chasing a phantom.
  • eBPF attaching to kernel functions. If the symbol or BTF source doesn't match the running kernel's layout, the program attaches to the wrong place or refuses to load — the same map-vs-reality mismatch, one layer up.
  • GPU / inference profiles. A trace that blames "kernel X" is only as trustworthy as the toolchain symbols behind it; a CUDA version skew turns a profile into confident fiction.

The pattern to keep: when a debugger insists nothing is happening, suspect the map before the code. A tool that reports the wrong address with total confidence is more dangerous than one that crashes, because you'll believe it. Aligning the map — nokaslr here, matched build IDs elsewhere — is the step that turns the tool back into a source of truth.

Related

Top comments (0)