DEV Community: 7mo

Secure Ransomware Development 101 ~ How Do malware researchers break your ransomware via faulted compiler optimization

7mo — Thu, 19 Mar 2026 22:19:45 +0000

so if you haven't seen part 1 you must go and check it out , so you can understand how does optimization influence code and what type of security issues does it bring to life .

lets dive strait into a simple example of a text encryption that is flawed not by the code , but the compiler optimization .



#include <stdio.h>
#include <string.h>
#include <stdlib.h>

void _FALWEDcleanup (char *ptr, size_t len) {if (ptr) { memset(ptr, 0, len); free(ptr);}}

void _CRYPTR (char *data, const char *key) { 
    size_t inputLength = strlen(data);
    size_t keyLen = strlen(key);

    for (size_t i = 0; i < inputLength; i++) { 
        data[i] = data[i] ^ key[i % keyLen];
    }
}

int main(void) { 
    size_t bufferSize = 100;
    char *input = malloc(bufferSize);

    if (input == NULL) {
        return 1;
    }

char *ourKEY = malloc(20);
strcpy(ourKEY, "i_love_you_000x");

    printf("enter some text to encrypt :  ");
    if (fgets(input, (int)bufferSize, stdin)) {
        input[strcspn(input, "\n")] = 0; 

        size_t len = strlen(input);

        _CRYPTR(input, ourKEY);
        printf("results (hex so you can see it ): ");
        for (size_t i = 0; i < len; i++) {
            printf("%02X ", (unsigned char)input[i]); 
        }
        printf("\n");

        _CRYPTR(input, ourKEY);
        printf("Decrypted: %s\n", input);

        // here is the error we are going to analyze
        _FALWEDcleanup(ourKEY, strlen(ourKEY));
    }

    return 0;
}

as you see this is a very simple exmaple
we simply take in a user input XOR it with a key then print out the output , then xor it again that will reverse the operation and print out the decrypted text

if you take a look at line 44 = we call a cleaner function which is doing :

void _FALWEDcleanup (char *ptr, size_t len){
if (ptr) {
 memset(ptr, 0, len);
 free(ptr);}
}

so in English this code is " writes zeros over our data to hide it and then tells the computer you're done using that memory. "

so the developer is hoping that by setting it to zeros memory dumps wont show the key which is absolutely wrong if you optimize your code .

lets first take a look at a disassembly of the non optimized code (without using the -O3)

as we see the cleanup function is there when we didn't optimize the code now lets check if its still there if we optimize the code with ' -O3 '

as you see the cleanup function has been completely removed !!!

so in theory this means the key will still reside in the memory ?

(note :

While a simple example might use a hardcoded key—which you could easily extract using tools like Microsoft Strings or flare FLOSS—real-world ransomware is much more sophisticated. In a live attack, the symmetric encryption key is typically generated at runtime, used to encrypt files, and then immediately wiped from memory or encrypted with an attacker’s public key to prevent recovery through memory forensics.

and we are looking at the bugs that cause the key not to be fully wiped so we can fix them )

~ BREAKING THE ENCRYPTION VIA MEMORY DUMPS ANALYSIS

so before we start just edit the example code and append
" getchar(); "
at the end so we can capture the memory after everything has finished instead of the program just terminating

to dump the memory of a process you can use the following command :

sudo gcore -o (output_file)

and to get the process id you can simply use pgrep " pgrep "

so lets go ahead and compile & run the program with optimization :
gcc -o code .c -O3

then enter a string when it asks you and wait until it stops at the get char then we will use

pgrep code

then we will do

sudo gcore DUMP.DUMP

then you can run strings and grep for our key and you should 100/100 percent see it !!!

and if you try without optimization you wont be able to see it since its zeroed out

REEFER TO MY VIDEO WHERE I EXPLAIN THE CONCEPTS :

note this is my first video and lowkey i cant speak, LOL. i have a mistake saying that the string is being copied to the pointer while it is actually being copied to the memory address that the pointer is looking at, but pointers are defined as " a variable that stores the memory address of another variable, rather than a direct value " so in a way im correct but yk its better to say "memory address " .......

the video has a small delay (1.8 seconds)

Secure Ransomware Development 101 ~ understanding compiler optimizations

7mo — Thu, 19 Mar 2026 08:39:29 +0000

PLEASE NOTE: Despite the provocative title, this article is strictly an educational deep-dive into ransomware internals and memory forensics. I’m showing you how the tech works under the hood—I am NOT telling you to develop ransomware or do anything illegal.

In this blog post, we’re going to explore Compiler Optimizations—what they are, how they work, and why sometimes they might not be as helpful as you think for your code.
So, let’s start with the basics: what exactly is compiler optimization, and how does it work?
note : we use

-O3 in this article

_

1 ~ what in the world is compiler optimization ?

in the design theory of it the fellow developers noted that the main goal of CO , is to make the code " more efficient, without changing its original functionality " and of course the primary goals are typically to improve execution speed, reduce memory usage, minimize code size, or lower power consumption.

now lets take a look at how do they work

2 ~ compilation pipeline

OK cool now you understand how does the pipeline look like , but i focused at the ir (intermediate representation) optimization specifically (DEAD STORAGE ELIMINATION) since its the main topic of this blog .

3 ~ taking a look at how does the compiler work with the -O3 flag and what affects does it do on the code .

ok , lets see this through a disassembler to understand whats really happening .

take this example first :

#include <stdio.h>
#include <string.h>

int secret(int *ptr) {
    return *ptr;
}

int main() {
    int data = 42;
    int *y = &data;
    int val = secret(y);
    memset(&data, 0, sizeof(data));
    //printf("value was: %d, and now after memset: %d\n", val, data);
    return 0;
}

gef➤  disas main
Dump of assembler code for function main:
   0x0000000000001040 <+0>: endbr64
   0x0000000000001044 <+4>: xor    eax,eax
   0x0000000000001046 <+6>: ret
End of assembler dump.
gef➤

as you see nothing if this asm was code it will be just a return 0; as you see it xor eax to return zero and ret .

because there is no output from this program

first lets see the Dead Store Elimination (DSE)

This targets writes to memory.
The "Store": memset(&data, 0, ...) is a command to write zeros to memory.
Why it's "Dead": Because the variable data is never being used again after that line. The compiler sees you writing a value that will never be used by anyone.

Result: The compiler deletes the memset call
entirely.

now lets take a look at Dead Code Elimination (DCE)
This targets computations and logic that don't affect the program's output.

The "Code": The variable val = secret(y) and the initialization int data = 42.

Why it's "Dead": Once we commented out the printf, the variable val became useless. It doesn't influence the return value of main (which is hardcoded to 0), and it doesn't trigger any I/O.
Result: The compiler deletes the secret function call and the int data variable itself.

\
what happens if there was a printf ? :

something very interesting happens , fr your going to be surprised by how good are now day compilers in optimizing code

now lets check it with a printf

#include <stdio.h>
#include <string.h>

int secret(int *ptr) {
    return *ptr; 
}

int main() {
    int data = 42;    
    int *y = &data;   

    int val = secret(y); 


    memset(&data, 0, sizeof(data)); 

    printf("value was: %d, and now after memset: %d\n", val, data); 

    return 0;
}


   0x0000000000001060 <+0>: endbr64
   0x0000000000001064 <+4>: sub    rsp,0x8
   0x0000000000001068 <+8>: xor    ecx,ecx
   0x000000000000106a <+10>:    mov    edx,0x2a
   0x000000000000106f <+15>:    mov    edi,0x2
   0x0000000000001074 <+20>:    lea    rsi,[rip+0xf8d]        # 0x2008
   0x000000000000107b <+27>:    xor    eax,eax
   0x000000000000107d <+29>:    call   0x1050 <__printf_chk@plt>
   0x0000000000001082 <+34>:    xor    eax,eax
   0x0000000000001084 <+36>:    add    rsp,0x8
   0x0000000000001088 <+40>:    ret
****

the optimized assembly shows that the compiler removed all of that code. Instead of actually executing those functions at runtime, it realized that secret(y) always returns 42 and that data is only used in the subsequent printf. Since memset(&data, 0, sizeof(data)) simply sets data to zero and nothing else uses it, the compiler replaced the values directly in the printf call. In other words, the instructions for assigning 42, calling secret(), and zeroing data are gone — everything is hardcoded constants passed straight to printf .

thats amazing tbh , cool asl .

But how can some compiler optimization cause some security issues ?

4 ~ security issues !!

#include <string.h>
#include <stdio.h>

void process_login() {
    char password[64];


    strcpy(password, "_thx_for_reading"); 


    printf("عيد مبارك \n");


    memset(password, 0, sizeof(password)); 
} 

int main() {
    process_login();
    return 0;
}

the problem with this code if it has been compiled with optimizations the memset that zeros out the buffer in the memory will just be optimized under the rule that nothing after the memset touches the buffer again .

whats the impact ? :

1 : If our program has a separate bug (like a buffer over-read), an attacker can trick the program into sending back more data than it should. Since the password was never wiped, it's just sitting in the "garbage" area of the RAM, ready to be leaked in a response.

2 : core dumps and memory analysis

3 : If another function runs right after process_login, it will re-use the same stack space.
If that second function has a vulnerability (like a "format string" bug), an attacker can peek at the local variables of that function.
Because the memory wasn't cleared, they aren't just seeing the new function's data—they are seeing the leftovers of our password.

see you in part 2 .... : )

Race to the Bottom: Weaponizing Concurrency , PART 1 . #THREADING

7mo — Sat, 17 Jan 2026 02:34:25 +0000

btw im learning while creating this blog so if there is any mistake please tell me, and i will fix it and learn , THANKS !

HI i like to dive into things immediately so take a look at this code sample here :

CHAPTER 1 : THREADS ?

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <arpa/inet.h>

#define PORT 8080

void handle_request(int client_socket) {
    char response[2048];

    strcpy(response, "HTTP/1.1 200 OK\r\nContent-Type: text/html\r\n\r\n");
    strcat(response, "<html><body><h1>Hi</h1>");
    write(client_socket, response, strlen(response));

    sleep(3);

    memset(response, 0, sizeof(response));  
    strcat(response, "<h1>Hi again!</h1></body></html>");
    write(client_socket, response, strlen(response));

    close(client_socket);
}

/*-------------------------------------------------------------------------------------------*/

int main() {
    int server_socket, client_socket;
    struct sockaddr_in server_addr, client_addr;
    socklen_t client_len = sizeof(client_addr);

    server_socket = socket(AF_INET, SOCK_STREAM, 0);
    if (server_socket < 0) {
        perror("Error creating socket");
        exit(1);
    }

    server_addr.sin_family = AF_INET;
    server_addr.sin_addr.s_addr = INADDR_ANY;
    server_addr.sin_port = htons(PORT);  

    if (bind(server_socket, (struct sockaddr *)&server_addr, sizeof(server_addr)) < 0) {
        perror("Error binding");
        exit(1);
    }

    if (listen(server_socket, 5) < 0) {
        perror("Error listening");
        exit(1);
    }

    printf("Server listening on port %d...\n", PORT);

    while (1) {
        client_socket = accept(server_socket, (struct sockaddr *)&client_addr, &client_len);
        if (client_socket < 0) {
            perror("Error accepting connection");
            continue;
        }

        handle_request(client_socket);
    }

    close(server_socket);
    return 0;
}

in the handle function ,
it has a response buffer that holds 2048 bytes ,
then it copies 200 ok and Content type into the response array after that it adds a message that says HI in the web page then writes it .
sleep 3 means wait 3 seconds and then show me another message that simply says hi again after that it closes the connection .

--- # ---
What could go wrong? The main issue with this code is that it is not threaded. This means the server processes each request sequentially — it handles one request at a time. The server blocks on each request, meaning it cannot process a new request until the current one is fully completed. As a result, the server essentially "locks itself up" while waiting for each request to finish. This approach becomes very inefficient, especially under heavy load, leading to poor performance when multiple clients try to connect at the same time. This is slow!

and also can be exploited to cause a DOS and brick the server ? how lets get into it !

CHAPTER 2 :CAUSING THE DOS

AFTER compiling the code as you can go to :
**

http://127.0.0.1:8080

yeah its working cool !
but as you saw when you went it takes 3 seconds then prints hello again but the real question is how can we see the no thread thing ?
open multiple tabs instances of the web app on the left and one on the right click them all and as you see the one on the right doesnt only take 3 seconds it takes much more time ?
why ? ?? ?????
welp simply because the server waits up until it process each one of them then goes to the last requested one

if your too into it you can debug and see how does it like pause like wait until it process the first one

but no need running man accept tells you that

The accept() system call is used with connection-based socket types (SOCK_STREAM, SOCK_SEQPACKET). It extracts the
first connection request on the queue of pending connections for the listening socket

as you see it has a queue takes one process then close . yeah pretty slow !

CHAPTER 3 : THREADING IT !

first what is threading ?

Threading in coding refers to the ability of a program to run multiple operations (or threads) concurrently. It allows a program to perform tasks in parallel, which can make it run more efficiently, especially when dealing with time-consuming operations like I/O (input/output) tasks, or heavy computations.

Thread: A thread is the smallest unit of execution in a program. A thread is part of a process and can run concurrently with other threads, but each thread has its own instruction pointer, stack, and local variables.

Multithreading: This is the technique of running multiple threads at the same time within a single program. Threads share the same memory space, which allows them to communicate easily with each other but also means that you have to manage shared data carefully to avoid conflicts.

Why use threading?

Concurrency: While a single-threaded program can do only one thing at a time, a multithreaded program can perform several tasks in parallel, improving efficiency.

Responsiveness: For GUI applications, using threads can keep the user interface responsive, even while the program is working on background tasks.

Utilizing Multi-core Processors: Modern CPUs have multiple cores, and threading allows programs to use these cores for parallel processing.

take a look here :

this image is nice in visualizing the concept when multi threading the code all threads within the same process share the code, data, and files. This makes communication between threads very fast because they can all "see" the same global variables. and of course each one of them has its own stack and registers

before moving on to making our server multi threaded i recommend you to take a look at this blog :

Multithreading in C - GeeksforGeeks

Your All-in-One Learning Portal: GeeksforGeeks is a comprehensive educational platform that empowers learners across domains-spanning computer science and programming, school education, upskilling, commerce, software tools, competitive exams, and more.

geeksforgeeks.org

written by kartik

--------- # -------------
so lets get into making our code multi threaded and explaining how does it work :

In C programming language, we use the POSIX Threads (pthreads) library to implement multithreading, which provides different components along with thread management functions that create the foundation of a multithreaded program in C.

Creating a Thread
The first step is to create a thread and give it a task. To create a new thread, we use the pthread_create() function provided by the thread library in C. It initializes and starts the thread to run the given function (which specifies its task).

Syntax

pthread_create(thread, attr, routine, arg);
where,

thread : Pointer to a pthread_t variable where the system stores the ID of the new thread.
attr : Pointer to a thread attributes object that defines thread properties. Use NULL for default attributes.
routine: Pointer to the function that the thread will execute. It must return void* and accept a void* argument.
arg: A single argument passed to the thread function. Use NULL if no argument is needed. You can pass a struct or pointer to pass multiple values.

SEEMS EASY !

we only need to simple changes :

1 -- > append the library header : #include .
2 -- > second we go back to filling the pthread_create() params

------ filling the arguments ------
looking at the man page of pthread_create the first argument is : pthread_t *restrict thread which also explained in man is :

---------------------------------------------------------------
Before  returning,  a  successful call to pthread_create() stores the ID of the new thread in the buffer pointed to by
       thread; this identifier is used to refer to the thread in subsequent calls to other pthreads functions.
-----------------------------------------------------------

so we declare it by adding in the code : pthread_t thread_id;

second argument

 The  attr  argument  points to a pthread_attr_t structure whose contents are used at thread creation time to determine
       attributes for the new thread; this structure is initialized using pthread_attr_init(3)  and  related  functions.   If
       attr is NULL, then the thread is created with default attributes.

so we leave our second argument as null

third argument : in the function syntax it says void (*start_routine)
so start routine most likely means the function and our function is simply the handler but looking at the blog written by kartik it says :
**_routine: Pointer to the function that the thread will execute. It must return void and accept a void* argument._

so first we want to return a void pointer so in the function end it with

return NULL;
2 -- > IT MUST ACCEPT A VOID ARGUMENT SO I SIMPLY DID :
void *client_socket_ptr

so after fixing things casting and calling we are done and left with :

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <arpa/inet.h>
#include <pthread.h> 

#define PORT 8080

void *handle_request(void *client_socket_ptr) {
    int client_socket = *((int *)client_socket_ptr);  
    char response[2048];

    printf("Handling request...\n");

    strcpy(response, "HTTP/1.1 200 OK\r\nContent-Type: text/html\r\n\r\n");
    strcat(response, "<html><body><h1>Hi</h1>");
    printf("Response sent: Hi\n");
    write(client_socket, response, strlen(response));

    sleep(3);  

    printf("Waiting for 3 seconds...\n");

    memset(response, 0, sizeof(response));  
    strcat(response, "<h1>Hi again!</h1></body></html>");
    printf("Response sent: Hi again\n");
    write(client_socket, response, strlen(response));

    close(client_socket);
    printf("Client socket closed\n");

    return NULL;  
}

/*-------------------------------------------------------------------------------------------*/

int main() {
    int server_socket, client_socket;
    struct sockaddr_in server_addr, client_addr;
    socklen_t client_len = sizeof(client_addr);

    printf("Starting server...\n");

    server_socket = socket(AF_INET, SOCK_STREAM, 0);
    if (server_socket < 0) {
        perror("Error creating socket");
        exit(1);
    }
    printf("Server socket created\n");

    server_addr.sin_family = AF_INET;
    server_addr.sin_addr.s_addr = INADDR_ANY;
    server_addr.sin_port = htons(PORT);  

    if (bind(server_socket, (struct sockaddr *)&server_addr, sizeof(server_addr)) < 0) {
        perror("Error binding");
        exit(1);
    }
    printf("Socket bound to port %d\n", PORT);

    if (listen(server_socket, 5) < 0) {
        perror("Error listening");
        exit(1);
    }
    printf("Server listening on port %d...\n", PORT);

    while (1) {
        printf("Waiting for client connection...\n");

        client_socket = accept(server_socket, (struct sockaddr *)&client_addr, &client_len);
        if (client_socket < 0) {
            perror("Error accepting connection");
            continue;
        }

        printf("Client connected, handling request...\n");

        pthread_t thread_id;
        if (pthread_create(&thread_id, NULL, handle_request, (void *)&client_socket) != 0) {
            perror("Error creating thread");
        }

        pthread_detach(thread_id);
    }

    close(server_socket);
    printf("Server socket closed\n");

    return 0;
}

as you see super simple changes !
recompile and run it
access it from multiple browsers and tabs do whatever you see it always takes 3 seconds and your hi again is there !

BUT !!!! (the way) this code has been developed has a security vulnerability in it , see you in part 2 ;) -- we will have a fun time debugging and understanding and finding such cool issues !