Reverse Engineering Undocumented Architectures: LLM-Driven Opcode Table Extraction vs. Legacy Tooling Constraints

Executive Summary

When building a custom disassembler for an undocumented, legacy, or modified CPU architecture, the primary challenge is not writing the decoding loop, but obtaining a structured, machine-readable opcode-to-instruction mapping table.

While one might expect to extract these tables from existing reverse-engineering frameworks like Ghidra, doing so is surprisingly difficult. Ghidra’s instruction definitions are deeply integrated into its SLAgh engine and designed for internal P-Code translation rather than data export. Traditional CLI disassemblers (like dis51) hardcode their mappings directly inside compiled execution logic.

This article presents an alternative methodology: leveraging a Large Language Model (Gemini) to reconstruct, normalize, and export a complete opcode lookup table into a clean Python structure in minutes, and using it to power a lightweight linear sweep disassembler.

---

1. The Core Problem: Why Extracting Opcode Tables from Existing Tools Fails

To build a lightweight, custom disassembler, you need a simple lookup table:

Opcode Byte ──> (Length, Mnemonic, Operand Formatting)

However, existing toolchains make retrieving this structured table nearly impossible:

• Ghidra (SLAgh Engine Integration):
  Ghidra holds extensive architecture data within its .slaspec files. However, these files are written in SLAgh—a complex domain-specific language designed to compile instruction logic into Ghidra’s internal P-Code representation. Ghidra offers no straightforward API or export utility to extract a flat opcode mapping table from these definitions without parsing SLAgh grammar or running heavy Java context frameworks.

• Dedicated Disassemblers (dis51):
  Standalone tools do not store opcode maps as external data files. The byte decoding, instruction length calculation, and operand formatting are hardcoded directly into C/C++ control-flow logic.

• Datasheets and Specifications:
  Manufacturer documentation presents opcode tables in fragmented PDF tables, diagrams, and text descriptions, requiring hours of manual, error-prone data entry.

---

2. Experimental Protocol & Testing Baseline

To validate our LLM-based table generation against a ground-truth baseline:

1. Test Image Generation (test1_8051.rom):
   A target program (test1.c) containing mixed 8/16/32-bit arithmetic, conditional branches, switch jump tables, and embedded literal strings ("TARGET_STRING") was compiled using SDCC (-mmcs51). The result was converted into a raw binary ROM image containing both code (crt0, math helpers) and read-only data blocks.

2. The Black-Box Experiment:
   We treated test1_8051.rom as an unknown architecture binary to evaluate how different disassembly strategies perform when full control-flow entry points are not pre-configured.

---

3. Disassembly Behavior & Tooling Benchmark

Evaluating the binary across different disassembly methods highlighted distinct architectural limits:

Tool / Approach	Method for Opcode Mapping	Disassembly Coverage	Main Limitation
Ghidra	Requires SLAgh (.slaspec) definitions	N/A (Requires setup)	Cannot export raw opcode maps; requires full framework setup.
dis51	Hardcoded C logic	Incomplete (Stalls)	Stops at RET / helper functions / inline strings ("TARGET_STRING").
Custom Python (Our Approach)	Gemini LLM Reconstruction	100% (Linear Sweep)	Ignores control flow; requires manual identification of data vs. code.

Why dis51 Fails on Raw Binaries

Because dis51 uses recursive traversal (tracing execution paths from 0x0000), it relies on detecting jump/call boundaries. When it hits library return instructions (RET), runtime math helpers, or inline data blocks (such as "TARGET_STRING" at 0x02CA), tracing halts. Unreached code sections are prematurely categorized as raw bytes (.DB).

---

4. Pipeline: LLM-Driven Opcode Extraction & Linear Sweep

By shifting the role of the LLM from "disassembling code" to "extracting and structuring the opcode table", we decouple data extraction from execution logic.

[ Raw Specification / PDF / Prompt ] ──> [ Gemini LLM ] ──> [ Python INSTRUCTION_TABLE ]
                                                                       │
[ Raw Binary Image (.rom) ] ─────────────> [ Linear Sweep Engine ] ────┴─> [ Assembly Output ]

Step 1: Opcode Normalization via Gemini

Instead of parsing Ghidra’s SLAgh files or writing complex XML parsers, we prompt Gemini directly to extract, deduplicate, and structure the architecture's instruction set from raw reference data.

We used the following structured prompt to enforce a strict, semicolon-delimited CSV-like format:

Prompt sent to Gemini:
"Generate an output file where each line follows this exact structure:
1) The number of bytes in the instruction (e.g., 3)
2) The instruction opcode bytes in hexadecimal (e.g., 75 81 18)
3) The disassembled assembly instruction (e.g., MOV SP, 18h)
4) A column with the generic instruction pattern (e.g., MOV register, value8)
Use a semicolon (;) as the field delimiter.
Example 1:
02 30 00 must become: 3;02 30 00;LJMP 0030;LJMP addr16
Deduplication rule:
If two consecutive lines share the exact same value in the last column and that pattern does **not* contain the word register, omit the second line from the generated file.*
Example 2:
If the following two entries are generated:
3;02 30 00;LJMP 0030;LJMP addr16
3;02 30 01;LJMP 0130;LJMP addr16
Do **not* include the second line in the final output."*

The raw output generated by the LLM is then directly converted into a lightweight, standalone Python lookup dictionary used by our disassembler:

# LLM-Generated Mapping Table (parsed from structured output)
INSTRUCTION_TABLE = {
    0x00: (1, "NOP", None),
    0x02: (3, "LJMP", "addr16"),
    0x12: (3, "LCALL", "addr16"),
    0x75: (3, "MOV", "register, value8"),
    0x22: (1, "RET", None),
    0x54: (2, "ANL", "A, #data"),
    # ... fully populated and deduplicated by Gemini
}

Step 2: Linear Sweep Disassembly Engine

A minimal Python script sweeps through the binary sequentially:

1. Reads the lead byte as an opcode index.

2. Fetches instruction length, mnemonic, and operand rules from INSTRUCTION_TABLE.

3. Formats operands and advances the offset pointer linearly.

---

Step 3: Comparative Analysis & Family Identification (Differential Mapping)

The true power of generating a standardized, machine-readable opcode table via LLM lies in cross-architecture correlation. Once a partial or candidate opcode table is constructed from an unknown binary, it can be diffed against a reference database of known ISA (Instruction Set Architecture) tables.

[ Candidate / Partial Table ] ──┐
├──> [ LLM / Script Diff Engine ] ──> 1. Identify Core Architecture
[ Known ISA Database (8051, ] ──┘                                     2. Infer Missing Opcodes
Z80, RISC-V, PIC...)                                                3. Highlight Custom Extensions

This differential analysis enables three critical reverse-engineering breakthroughs:

1. Architecture Classification (Fingerprinting):
   
   Even heavily customized microcontrollers usually retain structural heritage from legacy families (e.g., MCS-51, Z80, or AVR). By comparing opcode frequency and instruction layout, the LLM can instantly recognize the underlying base architecture.

2. Inference of Unobserved Opcodes:
   
   If an unknown binary only utilizes 60% of the processor's opcode map, comparing the partial table against known variants allows us to predict the meaning and operand structure of the remaining 40% unobserved instructions.

3. Isolating Proprietary Opcodes:
   
   Any opcode that deviates from standard reference tables immediately highlights hardware-specific customizations, undocumented vendor extensions, or custom security mechanisms.

---

5. Results & Conclusion

1. Ghidra and legacy disassemblers lock away opcode knowledge: They are built to execute disassembly, not to expose their internal mapping tables as clean, reusable datasets.

2. LLMs solve the table generation bottleneck: Gemini bridges the gap between raw, unstructured datasheets and structured Python lookup tables in minutes.

3. Linear sweeping guarantees full coverage: Sequential sweeping ignores control-flow traps (RET, jump tables, inline constants), ensuring every byte of an undocumented binary is decoded and exposed.

4. The need for a global opcode database: To analyze truly undocumented binaries at scale, the community must build a normalized, open-access database of opcode maps across all known architectures (from vintage CPUs to modern proprietary MCUs). By running automated differential matching between an unclassified binary's opcode distribution and this reference database, reverse engineers can instantly determine which processor family an unknown chip is derived from and accelerate hardware fingerprinting.