DEV Community

Cover image for i added opaque predicates, anti-debug, and section obfuscation to my PE packer published
iamsopotatoe
iamsopotatoe

Posted on

i added opaque predicates, anti-debug, and section obfuscation to my PE packer published

#cpp #security #lowlevel #windows

TinyLoad v4 is out — here's what i added and why it actually matters:

so TinyLoad v4 just dropped. if you don't know TinyLoad — it's my open-source PE packer for Windows. you throw an exe at it, it compresses and encrypts it with a custom VM, and spits out a self-extracting stub that runs the original entirely in RAM. no temp files, no installer, nothing written to disk. one .cpp file, no dependencies. here's the repo.

v3 was already a decent jump because that's when the custom VM came in — randomised opcode shuffling so every packed file speaks a different instruction set. v4 is more focused. three specific additions that each solve a different analysis problem:

  • VM opaque predicates
  • anti-debug checks
  • PE section name obfuscation

let me actually explain each one instead of just listing them.

opaque predicates — confusing static analysis

when someone's trying to reverse a packed binary statically, they're usually building a control flow graph — figuring out which code actually runs and in what order. an opaque predicate is a branch where the outcome is predetermined (always taken, or never taken) but looks like it could go either way without executing anything.

in v4 the VM bytecode that gets generated for decryption starts with this:

// load two constants into registers
eOp(bc,enc,LDI_I);  eR(bc,6); e64(bc,0x1337);
eOp(bc,enc,LDI_I);  eR(bc,7); e64(bc,0x1338);

// compare them — r8 will always be 1 since 0x1337 < 0x1338
eOp(bc,enc,CMP_I);  eR(bc,8); eR(bc,6); eR(bc,7);

// jump if nonzero — always jumps
eOp(bc,enc,JNZ_I);  eR(bc,8);
int opqPatch = (int)bc.size(); e32(bc, 0);

// this HLT is dead code — never reached
eOp(bc,enc,HLT_I);

// real decryption loop starts here
Enter fullscreen mode Exit fullscreen mode

the halt never actually executes. but a static analyser that doesn't evaluate the constants just sees a branch with two possible destinations — one being an immediate halt before any decryption happens. it has to either run the code to know the truth, or assume both paths are possible.

what makes this play nicely with the rest of TinyLoad is that opcodes are randomly shuffled at pack time. so 0x1337 and 0x1338 appear as completely different byte values in every packed file. no two samples look the same, which makes pattern matching across samples a lot harder too.

anti-debug — making dynamic analysis awkward

static analysis not working? fine, let's just run it in a debugger. v4 adds two checks before the loader does anything:

bool isDebugged() {
    if (IsDebuggerPresent()) return true;

    BOOL remote = FALSE;
    CheckRemoteDebuggerPresent(GetCurrentProcess(), &remote);
    if (remote) return true;

    return false;
}
Enter fullscreen mode Exit fullscreen mode

IsDebuggerPresent catches the obvious stuff — x64dbg, WinDbg attached locally. CheckRemoteDebuggerPresent goes a level deeper and catches kernel debuggers and remote sessions that the first check doesn't see.

if either one fires, the loader just returns false silently. no error, no crash, no messagebox. the packed exe runs and does absolutely nothing. that kind of silent failure is more disorienting than a visible error because it's not immediately obvious why nothing happened.

section obfuscation — breaking heuristic scanners

PE packers often get detected not from what the code does but from what the file looks like — and section names are one of the things scanners latch onto. if a packer consistently leaves behind a section called .tinyld or whatever the compiler decided to name things, that's a trivial signature.

after writing the packed output, v4 goes back into the PE headers and renames every section:

void scrambleSections(Bytes& data) {
    IMAGE_NT_HEADERS64* nt = (IMAGE_NT_HEADERS64*)(data.data() + dos->e_lfanew);
    IMAGE_SECTION_HEADER* sect = IMAGE_FIRST_SECTION(nt);

    const char* names[] = {".text", ".data", ".rdata", ".bss", ".idata"};
    for (int i = 0; i < nt->FileHeader.NumberOfSections; i++) {
        memset(sect[i].Name, 0, 8);
        strncpy((char*)sect[i].Name, names[i % 5], 8);
    }
}
Enter fullscreen mode Exit fullscreen mode

the payload itself lives in an overlay after the sections, not inside any of them, so renaming changes nothing about how the exe actually runs. but from the outside the file looks like any other plain Windows binary — .text, .data, .rdata. nothing to fingerprint.

putting it all together

grab the binary from releases or build it:

g++ -o TinyLoad.exe TinyLoad.cpp -static -O2 -s
Enter fullscreen mode Exit fullscreen mode

usage hasn't changed much from v3:

TinyLoad.exe --i myapp.exe --vm --c
Enter fullscreen mode Exit fullscreen mode

--vm is the custom VM encryption (now with the opaque predicates and anti-debug baked in). --c is LZ77 compression. compression runs first, then VM encryption goes on top — so the compressed stream's patterns get hidden too. you need at least one of the two flags.

if you find files it breaks on, open an issue. and if you use it, a star really does help ❤️

repo: github.com/iamsopotatoe-coder/TinyLoad
blog + changelogs: iamsopotatoe-coder.github.io/TinyLoad/#blog

Top comments (0)