DEV Community

Mirrai
Mirrai

Posted on

Buffer Overflows on x64 Windows: A Practical Beginners Guide (Part 1): Setting up

#beginners #infosec #security #tutorial

Introduction

Hello everyone. Mirrai here. I've been wanting to make this tutorial for a while because i feel guides on windows exploitation are hard to come across (especially x64) so i finally decided to start. This guide will walk you through the fundamentals of stack-based buffer overflows on x64 Windows, what they are, why they work, and how to set up your environment to start exploring them yourself.
By the end of part 1 you'll understand the stack, what RIP is and why controlling it allows arbitrary code execution and you'll have a vulnerable program ready to analyze. Part 2 will cover the actual, practical exploitation.

x86 vs. x64

If you’ve ever looked at older tutorials, you’ve probably seen a lot of talk about x86 (32-bit) exploitation. While the logic is similar, jumping to x64 (64-bit) feels like moving from an elevator to a warehouse. Everything is bigger and it will affect certain exploits later down the road.

In x86, your registers were 32 bits wide. In x64, they’ve doubled to 64 bits. This means every address you overwrite on the stack now needs to be 8 bytes long instead of 4. Certain registers such as EIP and ESP in x86 are now RIP and RSP in x64 as well.

x86 stack
0060F94C  00000000  
0060F950  00000000  

x64 stack
0000005189CFF0B0  0000000000000000  
0000005189CFF0B8  0000000000000000
Enter fullscreen mode Exit fullscreen mode

The Stack

Lets begin with the most basic and arguably hardest to understand part, the stack. When a program runs, it needs somewhere to temporarily store data, local variables, function arguments and return addresses. That place is the stack. Think of it as a pile of plates, uhh kinda. you add to the top and remove from the top. In x64 Windows, the stack grows downward in memory, meaning each new piece of data gets placed at a lower memory address than the previous one.

Random stack space
0000005189CFF908  0000000000000000  
0000005189CFF910  0000000000000000  
0000005189CFF918  0000000000000000  
0000005189CFF920  0000000000000000  
0000005189CFF928  0000000000000000  
0000005189CFF930  0000000000000000  
0000005189CFF938  0000000000000000  
0000005189CFF940  0000000000000000
Enter fullscreen mode Exit fullscreen mode

This is the stack memory from a random program i just ran. 0000005189CFF908 is the lowest section in this stack so if you need to store more data you do something like sub, rsp, 32 in ASM to allocate more below it such as 0000005189CFF900 and so on but I'm getting ahead of myself.

When a function is called, the program pushes a return address onto the stack. This is the memory address it should go back to when the function finishes.

What is RIP?

RIP is the instruction pointer register on x64 systems. It holds the address of the next instruction the CPU will execute. When a function returns, the value saved on the stack as the return address gets loaded into RIP. Wherever RIP points, execution follows.
Control RIP, control the program.

Say this is a function


00007FF6EB70139 | 48:83C4 28             | add rsp,28                                 
00007FF6EB70139 | C3                     | ret
Enter fullscreen mode Exit fullscreen mode

when ret is run the RIP changes to what rsp is pointed to.

RSP --> 0000005189CFF908  00007FF4EB801341  
        0000005189CFF910  0000000000000000
Enter fullscreen mode Exit fullscreen mode

If rsp is at 0000005189CFF908 then rip will change to and continue from there.

What is a Buffer Overflow?
A buffer is a fixed-size chunk of memory allocated for storing data. A buffer overflow happens when a program writes more data into a buffer than it was designed to hold, and that excess data spills into adjacent memory.
On the stack, that adjacent memory includes the saved return address. If you overflow far enough to overwrite the return address with a value you control, you control where execution goes when the function returns. That's the primitive.

Take this for example.

0000005189CFF810  0000000000000000  
0000005189CFF818  0000000000000000  
0000005189CFF820  0000000000000000  
0000005189CFF828  0000000000000000  
0000005189CFF830  0000000000000000  
0000005189CFF838  0000000000000000  
0000005189CFF840  0000000000000000  
0000005189CFF848  0000000000000000  
0000005189CFF850  0000000000000000  
0000005189CFF858  0000000000000000  
0000005189CFF860  00007FF6EF40131A
Enter fullscreen mode Exit fullscreen mode

Say i made this with

char buffer[80] = {0}; and that 0000005189CFF860 is our function's ret address.

If i write an "A" to this code using strcpy or whatever, I get:

0000005189CFF810  0000000000000041  
0000005189CFF818  0000000000000000  
0000005189CFF820  0000000000000000  
0000005189CFF828  0000000000000000  
0000005189CFF830  0000000000000000  
0000005189CFF838  0000000000000000  
0000005189CFF840  0000000000000000  
0000005189CFF848  0000000000000000  
0000005189CFF850  0000000000000000  
0000005189CFF858  0000000000000000  
0000005189CFF860  00007FF6EF40131A
Enter fullscreen mode Exit fullscreen mode

41 is "A" in hex btw and remember, the stack grows "down" (toward lower addresses) when you add more space to the stack but Buffers (like arrays) fill "up" (toward higher addresses). This is why the "A" is at 0000005189CFF810 but as we write more "A"s to this buffer, we will climb upward to 0000005189CFF860.

The goal is to overwrite 0000005189CFF860 so we need to write 88 "A"s. when I do that I get:

0000005189CFF810  4141414141414141  
0000005189CFF818  4141414141414141 
0000005189CFF820  4141414141414141  
0000005189CFF828  4141414141414141  
0000005189CFF830  4141414141414141  
0000005189CFF838  4141414141414141  
0000005189CFF840  4141414141414141  
0000005189CFF848  4141414141414141  
0000005189CFF850  4141414141414141  
0000005189CFF858  4141414141414141  
0000005189CFF860  4141414141414141
Enter fullscreen mode Exit fullscreen mode

Here we overwrote the ret address with "A"s which will crash the program because 4141414141414141 is not a real address that exists lol, but if we replace the value at 0000005189CFF860 with any other address we can control the RIP and run any code we want. Pretty cool right? Admittedly an actual function is a bit different to this but not by much. We will get there in the exploitation part.

Environment Setup

Now that we know the theory let's practice it. You'll need the following tools before proceeding:
x64dbg: A free, open source debugger for Windows. Download it at x64dbg.com. This is what you'll use to inspect memory, set breakpoints, and observe the overflow in action.

  • pwntools: A Python library for exploit development. Install it with: 
pip install pwntools
Enter fullscreen mode Exit fullscreen mode

Useful for scripting your exploit once you understand the mechanics manually.

  • A basic Python installation and a C compiler for building the vulnerable program round out the setup. I recommend winlibs for the gcc compiler plus MinGW-w64 with or without clang. You can get python here

The Vulnerable Program

The program we'll be exploiting is a simple C application with a classic stack vulnerability.

int main() {
   char username[20] = {0};
   printf("What is your username?: ");
   gets(username);
   printf("%s %s\n", "Hello", username);
}
Enter fullscreen mode Exit fullscreen mode

This program just greets you. How cute! Lets break it!

gets() is our entry point because it lacks bounds checking and will not stop if the data to be written is more than what the buffer can take. This is why it is advised to use fgets instead.

By the way, we will handle compilation and the arguments to pass to the compiler to disable certain protections like aslr, dep, stack canaries etc in part two. I might cover tutorials for getting around them sometime in the future.

What's Next

In part two we'll fire up x64dbg and pwntools, attach it to the vulnerable program, find the offset needed to control RIP, and build a working exploit.

See you all in part Two and of course if you have any questions feel free to ask.

Top comments (0)