DEV Community

ddupard
ddupard

Posted on

Identifying the Processor of a Bare-Metal Binary (Strategy 2): Modifying prompts

Industrializing the disassembly of an undocumented processor from a raw binary is a complex challenge that can be broken down into 4 main phases:

  • Verify that the binary does not belong to an already known processor.
  • Verify that the binary does not correspond to obfuscated, compressed, or encrypted code from a known processor.
  • Build an undocumented processor generator.
  • Develop the analysis workflow and disassembly generation pipeline.

For the first step of this project, the goal is to evaluate different strategies and identify the most effective approach.

2. Evaluated Strategies

The first strategy aimed to generate a transcodification table (mapping byte sequences to assembly instructions) in order to disassemble the binary both statically and dynamically for a given processor. This process continues until one or more bytes fail to match any known instruction for that processor, or until disassembly completes successfully (note: successful disassembly does not necessarily mean the binary was compiled for that specific processor).

To implement this strategy and build the transcodification table, several approaches were tested:
Ghidra transcodification table generation: Failed.
Native disassembler transcodification table generation: Failed.
Using Gemini to generate the table from raw byte sequences: Successful.

(For a full summary of the tests conducted under Strategy 1, see "Identifying the Processor of a Bare-Metal Binary (Strategy 1): Building a custom disassembler".)

The second strategy takes a completely different approach. It consists of using Ghidra to disassemble the binary against a large number of target architectures (177 processors) and then leveraging a Large Language Model (LLM) to analyze the resulting disassembly outputs to determine which processor truly matches the binary.

(For a full summary of the tests conducted under Strategy 1, see "Identifying the Processor of a Bare-Metal Binary — (Strategy 2): Testing LLMs".)

Today we are going to test if modifying the prompt of the model with the best results ( qwen3-coder:30b ) could lead to improved results.

3. Test Protocol

We are going to use the same binaries as in our preceding test.

  • Firmware 1: Target processor is natively supported by Ghidra.
  • Firmware 2: Target processor is not supported by Ghidra.
  • Firmware 3: An ELF file, explicitly disassembled as a raw bare-metal binary (forcing raw byte parsing).

The prompt 1 we use in our last test was

SYSTEM_PROMPT = """You are an expert in reverse engineering and processor architectures.
Your task is to verify the consistency of a raw firmware disassembly.
Examine the provided instructions, verify whether the architecture's syntax is valid,
and determine if the instructions appear coherent (absence of repeated invalid instructions, aberrant opcodes, etc.).
Respond strictly in valid JSON format."""
Enter fullscreen mode Exit fullscreen mode

In order to reduce the number of false positives, We will use the next 2 prompts.

Prompt 2:

SYSTEM_PROMPT = """You are a Principal Reverse Engineer specializing in bare-metal firmwares and undocumented ISAs.
Your goal is to REJECT invalid disassemblies and eliminate false positives.

Do NOT assume the code is valid just because instructions look syntactically correct.
Analyze the semantic coherence of the disassembly.

Follow this step-by-step reasoning (Chain-of-Thought):
1. CONTROL FLOW: Are there logical jumps/branches? Or is it a linear sequence of random instructions?
2. REGISTERS & STACK: Are registers used consistently? Is there a coherent function prologue/epilogue (stack handling)?
3. ABERRATIONS: Do you see repetitive opcodes, impossible immediate values, or meaningless instruction loops?
4. PATTERNS: Does this look like real compiled code or raw data/garbage interpreted as instructions?

Based on your analysis, fill out the following JSON. Be aggressive with disqualification: if in doubt, mark is_valid as false.

Respond STRICTLY in valid JSON with this schema:
{
    "reasoning": "Your step-by-step analysis detailing control flow, register use, and anomalies found",
    "processor_id": "{processor_id}",
    "is_valid": false,
    "confidence_score": 0.00,
    "detected_anomalies": ["list of specific anomalies, e.g., 'no jump targets', 'aberrant register R15 write'"]
}"""
Enter fullscreen mode Exit fullscreen mode

Prompt 3:

SYSTEM_PROMPT = """You are an expert in CPU architecture and reverse engineering.
Your task is to detect whether a disassembly snippet is REAL compiled code or GARBAGE output caused by wrong ISA decoding.

Example of GARBAGE code (Wrong ISA):
- Continuous stream of data movement without control flow (e.g. 50 MOV instructions in a row).
- Broken stack usage (e.g. POP without prior PUSH).
- Odd jump targets or incoherent register reuse.

Example of VALID code (Correct ISA):
- Structured prologues (e.g. push {r4-r7, lr}).
- Clear control flow (CMP followed by conditional BRANCH).
- Coherent stack frame adjustments.

Analyze the provided snippet and output STRICTLY a JSON format:
{
    "cot_analysis": "Step-by-step verification of prologues, jump density, and register logic",
    "processor_id": "{processor_id}",
    "is_valid": boolean,
    "confidence_score": float,
    "detected_anomalies": ["list of anomalies"]
}"""
Enter fullscreen mode Exit fullscreen mode

4. Benchmark Results

Firmware 1 (Known Processor)

  • Prompt 1
    qwen3-coder:30b: Generated reports for all 177 files. Identified 35 potential candidate processors (including the correct

  • Prompt 2

============================================================
                POTENTIAL CANDIDATES REPORT                
============================================================
 Model Tested  : qwen3-coder:30b
 Target Source : /home/daniel/Desktop/Assembleur/python/Toolchain/output_results
============================================================

[✓] Found 1 potential matching processor(s):

  • Processor  : 8051:BE:24:cip-51
    Confidence : 95.0%
    Summary    : The disassembly shows coherent 8051 code with logical control flow, proper function prologue/epilogue patterns, and consistent register usage. There are no obvious aberrations or garbage instructions. The code appears to be a valid firmware routine performing memory operations and I/O handling.
------------------------------------------------------------
============================================================
Enter fullscreen mode Exit fullscreen mode
  • Prompt 3 
============================================================
                POTENTIAL CANDIDATES REPORT                
============================================================
 Model Tested  : qwen3-coder:30b
 Target Source : /home/daniel/Desktop/Assembleur/python/Toolchain/output_results
============================================================

[✓] Found 17 potential matching processor(s):

  • Processor  : 6805:BE:16:default
    Confidence : 95.0%
    Summary    : The disassembly shows valid 6805 code with structured control flow using BRSET/BRCLR instructions, proper use of indexed addressing modes, and coherent register usage. No obvious garbage or malformed instructions are present.
------------------------------------------------------------
  • Processor  : PIC-17:LE:16:PIC-17C7xx
    Confidence : 95.0%
    Summary    : The disassembly shows valid PIC-17C7xx instructions with proper control flow, including CALL, GOTO, and RETURN operations. There are no obvious anomalies such as continuous MOV instructions or broken stack usage. The code appears to be structured with function calls and jumps, indicating real compiled code.
------------------------------------------------------------
  • Processor  : HC05:BE:16:default
    Confidence : 95.0%
    Summary    : The disassembly shows structured control flow with conditional branches, valid use of stack-relative addressing, and coherent register usage. No obvious garbage or malformed instructions are present.
...
Enter fullscreen mode Exit fullscreen mode

Firmware 2 (Unknown Processor)

  • Prompt 1
    qwen3-coder:30b: Generated reports for all 177 files. Identified 32 potential candidate processors (false positives).

  • Prompt 2

============================================================
                POTENTIAL CANDIDATES REPORT                
============================================================
 Model Tested  : qwen3-coder:30b
 Target Source : /home/daniel/Desktop/Assembleur/python/Toolchain/output_results
============================================================

[✗] No valid processor candidate found among the analyzed files.
============================================================
Enter fullscreen mode Exit fullscreen mode
  • Prompt 3 
============================================================
                POTENTIAL CANDIDATES REPORT                
============================================================
 Model Tested  : qwen3-coder:30b
 Target Source : /home/daniel/Desktop/Assembleur/python/Toolchain/output_results
============================================================

[✓] Found 12 potential matching processor(s):

  • Processor  : PIC-17:LE:16:PIC-17C7xx
    Confidence : 95.0%
    Summary    : The disassembly shows valid PIC-17C7xx instructions with coherent control flow and proper use of registers. There are no obvious anomalies such as broken stack usage or continuous data movement without control flow. The presence of NOPs at the end may indicate padding or optimization artifacts, but does not invalidate the code.
------------------------------------------------------------
  • Processor  : 8048:LE:16:default
    Confidence : 95.0%
    Summary    : The disassembly shows valid 8048 instruction set usage with structured control flow, appropriate use of flags (JTF, JF1, JB1), and coherent data movement. There are no obvious signs of garbage output such as continuous MOV instructions or broken stack usage.
------------------------------------------------------------
  • Processor  : z182:LE:16:default
    Confidence : 95.0%
    Summary    : The disassembly shows valid Z182 instruction set usage with structured control flow, appropriate use of registers, and logical sequence of operations including I/O operations and calls. No clear signs of garbage output or incorrect ISA decoding.
...
Enter fullscreen mode Exit fullscreen mode

Firmware 3 (ELF File disassembled as Bare Metal)

  • Prompt 1
    qwen3-coder:30b: Generated reports for all 177 files. Identified 42 potential candidate processors (including the correct one).

  • Prompt 2

============================================================
                POTENTIAL CANDIDATES REPORT                
============================================================
 Model Tested  : qwen3-coder:30b
 Target Source : /home/daniel/Desktop/Assembleur/python/Toolchain/output_results
============================================================

[✗] No valid processor candidate found among the analyzed files.
============================================================
Enter fullscreen mode Exit fullscreen mode
  • Prompt 3 
============================================================
                POTENTIAL CANDIDATES REPORT                
============================================================
 Model Tested  : qwen3-coder:30b
 Target Source : /home/daniel/Desktop/Assembleur/python/Toolchain/output_results
============================================================

[✓] Found 10 potential matching processor(s):

  • Processor  : PIC-17:LE:16:PIC-17C7xx
    Confidence : 95.0%
    Summary    : The disassembly shows valid PIC-17C7xx instructions with proper control flow and structured usage of registers. There are no signs of garbage output such as continuous data movement or broken stack usage.
------------------------------------------------------------
  • Processor  : 80390:BE:24:default
    Confidence : 95.0%
    Summary    : The disassembly shows valid 80390 instructions with coherent control flow and logical operation sequences. There are no obvious signs of garbage output such as continuous data movement or broken stack usage. The code includes valid instructions like RETI, JMP, and arithmetic/logic operations that follow expected patterns for this architecture.
...
Enter fullscreen mode Exit fullscreen mode

5. Can we improve the result of the last test ?

To address the failure in the third test (Firmware 3), we attempted to modify Prompt 2 to account for the potential presence of an ELF header or file metadata:

SYSTEM_PROMPT = """You are a Principal Reverse Engineer specializing in bare-metal firmwares, ELF containers, and undocumented ISAs.
Your goal is to REJECT invalid disassemblies while tolerance-checking for static header/metadata noise.

IMPORTANT FOR ELF/RAW BINARIES:
The first few instructions may contain header magic numbers (e.g. '\x7fELF' or vector tables) parsed as garbage assembly. 
Do NOT reject the entire disassembly immediately if only the initial instructions look like metadata/padding, PROVIDED that valid function logic or control flow starts shortly after.

Analyze the semantic coherence of the disassembly using this step-by-step reasoning (Chain-of-Thought):
1. NOISE VS CODE: Is the snippet pure, unrecoverable garbage throughout, OR does it transition into structured code after initial padding/header bytes?
2. CONTROL FLOW: Are there logical jumps, calls, or conditional branches in the main body of the snippet?
3. REGISTERS & STACK: Are registers used consistently, and is there coherent stack manipulation (push/pop, frame setup)?
4. ABERRATIONS: Do you see endless repetitive opcodes without any control flow, or impossible immediate values throughout the ENTIRE sample?

Decision Rule:
- REJECT (is_valid = false) if the disassembly is 100% random garbage, infinite repetitive MOV/NOP loops, or totally lacks control flow.
- ACCEPT (is_valid = true) if you detect a clear pattern of compiled machine code (prologues, jumps, loops), even if preceded by header noise.

Respond STRICTLY in valid JSON with this schema:
{
    "reasoning": "Your step-by-step analysis detailing header noise check, control flow, register use, and final verdict",
    "processor_id": "{processor_id}",
    "is_valid": false,
    "confidence_score": 0.00,
    "detected_anomalies": ["list of specific anomalies, e.g., 'initial ELF header noise detected but valid MIPS branches follow'"]
}"""
Enter fullscreen mode Exit fullscreen mode

However, using this new prompt, qwen3-coder:30b identified 39 potential matching processors for Firmware 3.

By instructing the model to tolerate initial header noise, we inadvertently destroyed the strict rejection bias that made Prompt 2 so effective. The model began using "header noise" as a universal excuse to justify false positives, finding imaginary structure in random byte decodings.

This result proves a fundamental rule in AI engineering: you cannot solve a data windowing problem using Prompt Engineering alone. The LLM should not be used to guess where the code begins; it must analyze code that has been properly delivered to it. So the pre processing pipeline should become

6. Epilogue

The results of this test are stunning. Modifying the prompt improved the result so incredibly. In our case, the prompt 2 gave us the correct answer for 2 of the 3 tests. This shows that the first thing to do when using a LLM is to find the best possible prompt. The last test showed us, that disassembling a binary should be done after having tested for headers, for compression, for obfuscation and encryption.
So now we have modify the toolchain in order to guaranty that the disassembling process occurs on a correct list of bytes.

Top comments (0)