DEV Community: TrothByte

You can't trust assembly an AI wrote. Here's the 3-command gate.

TrothByte — Sun, 16 Aug 2026 22:29:09 +0000

✨ Disclosure: this tutorial was drafted with AI assistance. Every technical claim is verified and source-traced in the linked repository.

AI assistants are confident about assembly — and wrong in specific, repeatable ways. They invent mnemonics that do not exist, flip AT&T/Intel operand order, silently drop immediates, and misread bytes. When we audited real incidents, the failures shared a signature: the code looks plausible and never errors.

The fix isn't "be more careful". It's a mechanical gate you can run in seconds.

The gate: assemble → disassemble → compare

# AT&T / GNU as, x86-64
gcc -c sample.s && objdump -d sample.o

# Intel syntax
gcc -c -masm=intel sample.s && objdump -d -M intel sample.o

If the disassembly does not match what you wrote — same mnemonic, same operands, same size — you did not write that instruction. Three real cases this catches:

1. A mnemonic that does not exist

movqad is not an instruction. The assembler rejects it — so far, so good. The dangerous ones compile.

2. A silently truncated immediate

imul eax, eax, 38      ; assembles to: 69 c0 00 00 00 00

The 38 is silently dropped by the parser — this is the bug class behind BBoeOS PR#584. The code compiles. The intent is gone.

3. A dialect swap

-masm=intel flips the operand order (mov eax, [rax] vs mov (%rax), %eax). Mixing AT&T and Intel in one file silently changes semantics.

Why "it compiles" is not enough

Compiling proves your syntax fits some grammar. It does not prove the encoding matches your intent. LLM-based disassembly gets exact instruction matches right about 14% of the time; "corrected" decompilations are right about 37%. The confidence-to-correctness gap is exactly where the expensive bugs live.

Rules to live by

Never assert an instruction, encoding, or length from memory. Assemble it.
Pin the syntax dialect explicitly. Never mix AT&T and Intel.
Check the raw bytes for unusual instructions against the manual (Intel SDM, ARM ARM, RISC-V ISA).
If the toolchain isn't available, say UNVERIFIED and give the command that would check.

The same discipline applies everywhere

Verify real parallelism (thread counts + wall time, not thread-safe syntax). Verify the API actually exists (cargo search, not memory). Verify your verification — a harness that can't fail is not evidence.

📦 The repository — actively updated

The full failure catalog and the skills that encode these gates live in
https://github.com/TrothByte/low-level-skills-trothbyte — 124 verified skills for
C, C++, Rust, assembly, kernel, embedded, Zig, GPU, reverse engineering, and build systems.

65 of 124 skills are executed on real toolchains (GCC 16.1, rustc 1.97, GDB, objdump, CMake/Ninja); the rest are honestly marked researched with exact verification commands.
Every claim is source-traced: claim → primary source → section → skill (177 primary sources).
The repository is continuously updated — new failure classes and skills are added as incidents are catalogued, and the whole library re-validates on every change:

git clone https://github.com/TrothByte/low-level-skills-trothbyte
python tools/validate.py     # 124 skills + registry + 177 sources, gated in seconds

Also installable via npx skills add TrothByte/low-level-skills-trothbyte or as a Claude Code plugin marketplace.

Found a failure we haven't catalogued? The repo accepts issues — every new skill must be source-traced and differentiated from the existing 124. Watch the repository to get updates as it grows.

We catalogued 55+ AI-agent failures in low-level code — and shipped 124 verified skills to fix them

TrothByte — Sun, 16 Aug 2026 22:26:06 +0000

✨ Disclosure: this article was drafted with AI assistance. Every technical claim in it is source-traced in the linked repository (registry/claims.yaml, 177 primary sources). The failure classes below come from real, documented incidents — not vibes.

AI coding agents are excellent at boilerplate and unreliable at low-level code — and the failures are not random. They cluster into predictable classes with a recognizable signature: the code looks correct, compiles, and still does the wrong thing.

We spent three research passes collecting 55+ documented failures (source-traced, not anecdotes) and turned them into 124 verified engineering skills. Here's what we learned.

TL;DR — the 5 failure classes

Assembly hallucinations — invented mnemonics, inverted operand order, silently truncated immediates.
Fake parallelism — thread-safe-looking code that runs on one thread.
Rust API drift & crate hallucination — behavioral API changes and nonexistent crates that resemble real ones.
Misleading verification — "passing" harnesses that never test the target.
Systems-level blind spots — VM leaks invisible to heap tools, timing side channels.

The fix isn't "be more careful". It's mechanical gates: assemble → disassemble → compare bytes; measure real parallelism; check the API actually exists; make your harness able to fail.

1. 🧠 Assembly hallucinations

Agents invent instructions that do not exist. One generated CDC COMPASS pseudo-ops (JOB, SST, OCT) for a program that "looked like assembly". Another produced movqad — no such instruction.

The dangerous failures don't error — they silently corrupt:

; What the agent wrote            ; What actually assembled
imul eax, eax, 38                  ; 69 c0 00 00 00 00  (the 38 is DROPPED)
mov (%rax), %eax                   ; 8b 00              (fine — but one byte vs. eax?)

imul eax, eax, 38 assembles to 69 c0 00 00 00 00 — the immediate is silently discarded by the parser. This is exactly the bug class behind BBoeOS PR#584. Add AT&T/Intel operand inversion, missing size hints (inc [counter] vs inc qword [counter]), and "AX is 8-bit" claims, and you have a reliable failure generator.

The gate:

gcc -c sample.s && objdump -d sample.o       # AT&T
gcc -c -masm=intel sample.s && objdump -d -M intel sample.o   # Intel

If the disassembly doesn't match what you wrote — same mnemonic, same operands, same size — you didn't write that instruction. Calibration: LLM disassembly gets exact matches right ~14% of the time; decompiler "fixes" are correct ~37%.

2. 🔁 Fake parallelism

Models produce ConcurrentHashMap/atomics that look thread-safe but execute everything on one thread. The CONCUR benchmark (arXiv:2603.03683) catches deadlocks and races that linear benchmarks cannot — because the code isn't actually concurrent.

Our gate is mechanical, not syntactic:

Program	Live threads	Wall-clock	Verdict
"thread-safe" demo	1	1.206 s	fake parallelism
real `pthread` split	4	0.304 s	real parallelism

Count live threads and measure wall-clock scaling. Thread-safe syntax is not parallelism.

3. 🦀 Rust API drift & crate hallucination

RustEvo² (arXiv:2503.16922) is the clearest dataset: models nail stabilized APIs at 65.8%, but behavioral changes (same signature, different semantics) at only 38%. Performance collapses from 56.1% to 32.5% for APIs added after the training cutoff. RAG helps (+13.5%) — but you can't RAG what doesn't exist yet.

And the supply chain angle is worse: agents hallucinate crates that do not exist but resemble real ones — a typosquatting risk (serde-json vs serde_json). Studies report 5.2% (commercial) to 21.7% (open-source) package hallucination rates.

cargo info serde-json    # exit 101 — does not exist
cargo info serde_json    # the real crate

The gate: verify existence (cargo search/crates.io API), pin the toolchain, and treat behavioral changes as the most dangerous class. In crypto specifically, only 23.3% of generated Rust compiles and 57% of that is vulnerable — with nonce reuse the leading cause (arXiv:2604.27001).

4. 🎭 Misleading verification

The worst class of all: a "passing" test that doesn't test the target.

A fixed-shape allclose oracle certifies buggy GPU kernels as correct — fuzz + fp64 reference catches 9/9 (arXiv:2606.20128).
Kernels "pass review, then segfault under load" (arXiv:2602.19594).
Ghostty's 37–130 GB VM leak: the page pool reused an mmap without ever calling munmap — an agent was the trigger, not the cause (mitchellh.com/writing/ghostty-memory-leak-fix).

The Iron Law: a harness that cannot fail is not evidence. The ablation test is: break the target — does your test catch it? If it still passes, your test is decoration.

5. 🧊 Systems-level blind spots

Timing side channels: early-exit string comparison leaks the matching prefix (CWE-1254; ~40% of Copilot's crypto code). Measured on our host: gcc -O2, 500k × 256-byte compares — early-exit memcmp 0.054 s vs constant-time ~0 s.
UB assumptions: "works at -O0, breaks at -O2" — the crash site is a symptom; the corruptor is elsewhere.

What we built: 124 verified skills

Each skill is a compact SKILL.md that answers five questions before a line is written:

Question	Why it matters
When to use / when not to	the agent loads the right tool, not everything
What the agent often gets wrong	the named failure classes above
How to reason correctly	the positive process, not just "don't"
What to verify / how	executable gates, not vibes
Where the knowledge comes from	every claim → primary source

Verified, not asserted: 65 of 124 skills were validated by actually running examples on real toolchains (GCC 16.1, rustc 1.97, GDB, objdump, CMake/Ninja). The remaining 59 are honestly marked researched with the exact command that would verify them.

git clone https://github.com/TrothByte/low-level-skills-trothbyte
python tools/validate.py     # 124 skills + registry + 177 sources, all gate in seconds

Also installable via npx skills add TrothByte/low-level-skills-trothbyte or as a Claude Code plugin marketplace.

The takeaway

The agents aren't broken — our expectations are. "It compiles" was never the bar for low-level code. The bar is: assemble → disassemble → compare bytes; measure real parallelism; check the API exists; make the test able to fail. The 55+ catalogued failures become 124 skills that encode exactly those gates.

If you write, review, or debug C, C++, Rust, assembly, kernels, or firmware with an AI — you'll recognize these failures. The library is free and MIT-licensed: github.com/TrothByte/low-level-skills-trothbyte.

Surveys with full source traces are in the repo's research/ folder. Found a failure we haven't catalogued? Open an issue — new skills must be source-traced and differentiated from the existing 124.