DEV Community

Eric-Octavian
Eric-Octavian

Posted on

24 hours of kernel debugging – keeping IONA OS alive in QEMU Lock ordering, backlight hangs, hypervisor detection, and an AMD FCH guard

Yesterday was a good day.

I fixed five bugs that had been bothering me for a while. None of them were catastrophic, but they were subtle — the kind of bugs that cause hangs, deadlocks, or silent failures that are hard to reproduce.

Here's what I fixed, how I found them, and what I learned.


1. Lock Order WM→BACK — The Deadlock You Don't See

The bug: IONA OS would occasionally hang when the window manager tried to acquire a lock that was already held by the backlight driver.

The problem: ABBA deadlock. Thread A locked WM, then tried to lock BACK. Thread B locked BACK, then tried to lock WM.

The fix: Reorder the locks. Always acquire BACK first, then WM. Consistent order = no deadlock.

// Before
lock(&WM);
lock(&BACK);

// After
lock(&BACK);
lock(&WM);

Simple, but hard to find without tracing.

  1. Backlight Skip — QEMU Has No Backlight The bug: On boot, IONA OS would hang at [BL] Backlight init... when running in QEMU.

The problem: QEMU doesn't have a backlight device. The driver was trying to communicate with hardware that didn't exist.

The fix: Detect the hypervisor and skip backlight initialization entirely.

if is_hypervisor() {
serial_println!("[BL] Hypervisor detected — skipping backlight init");
return;
}

Now IONA OS boots cleanly in QEMU.

  1. Hypervisor Detection — CPUID without the Flag The bug: is_hypervisor() would return false even in QEMU.

The problem: The CPUID 1 leaf ECX31 is not always set, especially with -cpu qemu64.

The fix: Don't rely on the flag. Probe the hypervisor leaf directly (0x4000_0000).

fn is_hypervisor() -> bool {
// Try the hypervisor leaf directly
let eax = unsafe { cpuid(0x4000_0000) }.eax;
eax != 0
}

This works with any QEMU configuration.

  1. EDP FCH Guard — MMIO That Doesn't Exist The bug: IONA OS would page fault when trying to read AMD FCH registers in QEMU.

The problem: The EDP driver was reading MMIO addresses that exist on real hardware (AMD FCH) but not in QEMU.

The fix: Add a guard — if the hypervisor is detected, skip the FCH access.

if is_hypervisor() {
serial_println!("[EDP] Skipping AMD FCH MMIO — running in VM");
return;
}

No more page faults.

  1. Disk Image Path — The Silent Fail The bug: The build pipeline was producing a disk image, but QEMU couldn't find it.

The problem: The build script was looking for x86_64-unknown-none/iona-pc.bin, but the actual path was x86_64-iona/debug/iona-pc.bin.

The fix: Update the build script to use the correct target directory.

Before

cp target/x86_64-unknown-none/debug/iona-pc.bin dist/

After

cp target/x86_64-iona/debug/iona-pc.bin dist/

A small fix, but it made the difference between booting and failing.

What I Learned

  1. Bug-finding is a skill. The hardest bugs aren't the ones that crash — they're the ones that silently hang or fail unpredictably.

  2. Hypervisor detection needs to be robust. Not all hypervisors set the standard flag. Always try the direct leaf.

  3. Lock ordering is critical. A single reversed lock order can cause intermittent deadlocks that are nearly impossible to reproduce.

  4. Even small fixes matter. Fixing a path in the build script can save hours of confusion.

The State of IONA OS
These five fixes are part of a larger effort to stabilise IONA OS

The kernel is now:

Bootable in QEMU and on real hardware.
Stable under load.
Actively maintained and debugged.

Resources
GitHub: github.com/Ionablokchain
Website: iona.zone

Top comments (0)