Secure Ransomware Development 101 ~ How Do malware researchers break your ransomware via faulted compiler optimization

so if you haven't seen part 1 you must go and check it out , so you can understand how does optimization influence code and what type of security issues does it bring to life .

---

lets dive strait into a simple example of a text encryption that is flawed not by the code , but the compiler optimization .

#include <stdio.h>
#include <string.h>
#include <stdlib.h>

void _FALWEDcleanup (char *ptr, size_t len) {if (ptr) { memset(ptr, 0, len); free(ptr);}}

void _CRYPTR (char *data, const char *key) { 
    size_t inputLength = strlen(data);
    size_t keyLen = strlen(key);

    for (size_t i = 0; i < inputLength; i++) { 
        data[i] = data[i] ^ key[i % keyLen];
    }
}

int main(void) { 
    size_t bufferSize = 100;
    char *input = malloc(bufferSize);

    if (input == NULL) {
        return 1;
    }

char *ourKEY = malloc(20);
strcpy(ourKEY, "i_love_you_000x");

    printf("enter some text to encrypt :  ");
    if (fgets(input, (int)bufferSize, stdin)) {
        input[strcspn(input, "\n")] = 0; 

        size_t len = strlen(input);

        _CRYPTR(input, ourKEY);
        printf("results (hex so you can see it ): ");
        for (size_t i = 0; i < len; i++) {
            printf("%02X ", (unsigned char)input[i]); 
        }
        printf("\n");

        _CRYPTR(input, ourKEY);
        printf("Decrypted: %s\n", input);

        // here is the error we are going to analyze
        _FALWEDcleanup(ourKEY, strlen(ourKEY));
    }

    return 0;
}

as you see this is a very simple exmaple
we simply take in a user input XOR it with a key then print out the output , then xor it again that will reverse the operation and print out the decrypted text

if you take a look at line 44 = we call a cleaner function which is doing :

void _FALWEDcleanup (char *ptr, size_t len){
if (ptr) {
 memset(ptr, 0, len);
 free(ptr);}
}

so in English this code is " writes zeros over our data to hide it and then tells the computer you're done using that memory. "

so the developer is hoping that by setting it to zeros memory dumps wont show the key which is absolutely wrong if you optimize your code .

lets first take a look at a disassembly of the non optimized code (without using the -O3)

as we see the cleanup function is there when we didn't optimize the code now lets check if its still there if we optimize the code with ' -O3 '

as you see the cleanup function has been completely removed !!!

so in theory this means the key will still reside in the memory ?

(note :

While a simple example might use a hardcoded key—which you could easily extract using tools like Microsoft Strings or flare FLOSS—real-world ransomware is much more sophisticated. In a live attack, the symmetric encryption key is typically generated at runtime, used to encrypt files, and then immediately wiped from memory or encrypted with an attacker’s public key to prevent recovery through memory forensics.

and we are looking at the bugs that cause the key not to be fully wiped so we can fix them )

---

~ BREAKING THE ENCRYPTION VIA MEMORY DUMPS ANALYSIS

so before we start just edit the example code and append
" getchar(); "
at the end so we can capture the memory after everything has finished instead of the program just terminating

to dump the memory of a process you can use the following command :

sudo gcore -o (output_file)

and to get the process id you can simply use pgrep " pgrep "

so lets go ahead and compile & run the program with optimization :
gcc -o code .c -O3

then enter a string when it asks you and wait until it stops at the get char then we will use

pgrep code

then we will do

sudo gcore DUMP.DUMP

then you can run strings and grep for our key and you should 100/100 percent see it !!!

and if you try without optimization you wont be able to see it since its zeroed out

---

REEFER TO MY VIDEO WHERE I EXPLAIN THE CONCEPTS :

note this is my first video and lowkey i cant speak, LOL. i have a mistake saying that the string is being copied to the pointer while it is actually being copied to the memory address that the pointer is looking at, but pointers are defined as " a variable that stores the memory address of another variable, rather than a direct value " so in a way im correct but yk its better to say "memory address " .......

• the video has a small delay (1.8 seconds)