DEV Community

Cover image for CTF Lab Writeup: "Bypass Me" — PicoCTF Binary Exploitation Challenge
Vedant Kulkarni
Vedant Kulkarni

Posted on

CTF Lab Writeup: "Bypass Me" — PicoCTF Binary Exploitation Challenge

1. Executive Summary

Field Details
Challenge Name Bypass Me
Category Binary Exploitation / Reverse Engineering
Difficulty Beginner–Intermediate
Target bypassme.bin on foggy-cliff.picoctf.net:53044
Primary Technique Static reverse engineering + XOR cipher decoding
Key Vulnerability Runtime password decoded via trivial XOR cipher, fully recoverable through static analysis
Tools Used ssh, file, nm, objdump, Python (mental model)
Flag Location Printed after successful password authentication

The Core Story

A binary that appears to be a secure, hardened authentication portal is completely defeatable through static analysis alone. The password is never truly hidden — it is simply obfuscated using a single-byte XOR operation. By reading the assembly, we recovered the password without ever running a debugger, without brute force, and without guessing. This challenge teaches one of the most fundamental lessons in security: obfuscation is not encryption.


2. Reconnaissance & Enumeration

2.1 Establishing Access

The challenge provides SSH credentials directly:

ssh ctf-player@foggy-cliff.picoctf.net -p 53044
# Password: 1ad5be0d
Enter fullscreen mode Exit fullscreen mode

Why SSH?

The challenge specifies it explicitly. SSH (Secure Shell) provides an interactive remote shell session, allowing us to explore, execute, and analyze the target binary in its native environment.

Breaking Down the Command

Flag Meaning
ctf-player The username we're logging in as
foggy-cliff.picoctf.net The remote hostname
-p 53044 Connect on port 53044 instead of the default SSH port 22

💡 Beginner Tip: Always accept the host fingerprint the first time you connect to a CTF server. In a real production environment, you would verify the fingerprint through a trusted, out-of-band method. In a CTF environment, it is generally safe to type yes.


2.2 Initial File System Enumeration

Once inside, the first thing we do is look around:

ls -la
Enter fullscreen mode Exit fullscreen mode

Why ls -la and not just ls?

  • -l → Long format: shows permissions, owner, size, and modification date
  • -a → Shows hidden files (those starting with .)

Output Analysis:

-rwsr-xr-x 1 root root 21672 Mar  6 20:10 bypassme.bin
Enter fullscreen mode Exit fullscreen mode

The most critical piece of information here is the permission string: -rwsr-xr-x.

That s where you'd normally expect an x for the owner execute bit is the SUID (Set User ID) bit. This deserves its own explanation:

🔑 What is SUID?
Normally, when you run a program, it runs with your permissions. When the SUID bit is set and the file is owned by root, the program runs with root's permissions, regardless of who launches it. This is how sudo and passwd work. In a CTF context, this tells us the binary can read files we cannot — like a flag file owned by root.

What we did NOT do:

  • We did not immediately try to exploit the SUID bit for privilege escalation (e.g., via PATH injection or LD_PRELOAD). Why? Because the challenge description told us the goal is to bypass authentication, not necessarily escalate privileges. The SUID bit is simply the mechanism by which the binary reads the flag file on our behalf.

2.3 Binary Profiling

file bypassme.bin
Enter fullscreen mode Exit fullscreen mode

Why file? The file command reads the magic bytes at the beginning of a file and identifies its true type regardless of extension. This is essential before analysis.

Output:

bypassme.bin: setuid ELF 64-bit LSB shared object, x86-64, dynamically linked, with debug_info, not stripped
Enter fullscreen mode Exit fullscreen mode

Parsing Every Field

Field Meaning & Significance
setuid ELF Confirms that the SUID bit is set. When executed, the binary runs with the privileges of its owner (in this case, root).
64-bit Indicates that the program uses a 64-bit architecture and register set (rax, rdi, rsi, etc.).
LSB Stands for Little-Endian Byte Order, the standard byte ordering used on most modern x86 systems.
x86-64 The binary targets the 64-bit Intel/AMD architecture. Most common reverse-engineering and debugging tools support this architecture natively.
dynamically linked The binary relies on shared libraries (such as libc) at runtime. Functions like strcmp() and printf() are typically resolved from external libraries.
with debug_info A valuable finding. Debug symbols are present, making reverse engineering and source-level debugging significantly easier.
not stripped Another valuable finding. Function names and other symbol information have been retained in the binary, improving readability during analysis.

💡 Key Insight: A "stripped" binary has had its symbol table removed, making reverse engineering significantly harder. A "not stripped" binary with "debug_info" is essentially giving us a roadmap. In a real-world malware scenario or hardened application, symbols would be stripped. This binary was compiled without stripping, which is a developer oversight.


2.4 Runtime Behavior Observation

Before touching any analysis tool, we always run the target to understand its user-facing behavior:

./bypassme.bin
Enter fullscreen mode Exit fullscreen mode

Why run it first? This gives us the "black box" view — what a normal user would see. We observed:

  • An ASCII art banner ("SECURE PORTAL")
  • A loading/initialization animation
  • A password prompt with a 3-attempt counter: [3 tries left] Enter password:

What this tells us:

  1. There is a finite attempt limit — brute force via the normal interface is impractical
  2. The password prompt means there's a comparison somewhere in the code
  3. The 3-attempt limit is enforced in software — meaning we can bypass it entirely by analyzing the binary rather than interacting with it

What we did NOT do:

  • We did not try common passwords (admin, password, 1234). That would be guessing, not hacking.
  • We did not use strings bypassme.bin as our primary approach. While strings can sometimes reveal hardcoded passwords, a developer using even basic obfuscation (like XOR) would defeat it. We wanted a methodology that works even when strings fails.

3. Initial Foothold — Static Reverse Engineering

3.1 Symbol Table Analysis

nm bypassme.bin
Enter fullscreen mode Exit fullscreen mode

What is nm? The nm command lists symbols from an object file or binary. Symbols are named entries in the symbol table — they include function names, global variables, and external library references.

Why nm before objdump? nm gives us the complete map of the binary's functions in seconds. It's the fastest way to understand the binary's structure before committing to deeper disassembly. Think of it as reading the table of contents before reading a book.

Key findings from nm output:

_Z15decode_passwordPc     decode_password(char*)
_Z13auth_sequencev        auth_sequence()
_Z8sanitizePKcPc          sanitize(char const*, char*)
_Z8type_outPKcj           type_out(char const*, unsigned int)
Enter fullscreen mode Exit fullscreen mode

💡 Name Mangling: The _Z15decode_passwordPc format is C++ name mangling. The compiler encodes type information into function names. _Z indicates a mangled name, 15 is the length of the function name, decode_password is the name, and Pc means "pointer to char". You can demangle these with c++filt _Z15decode_passwordPc.

The most important symbol:

_Z15decode_passwordPc   decode_password(char*)
Enter fullscreen mode Exit fullscreen mode

This immediately tells us: the password is not stored in plaintext. It is decoded at runtime by a function. This is our primary target.

External library calls that matter:

U strcmp@@GLIBC_2.2.5
Enter fullscreen mode Exit fullscreen mode

The U means "undefined" — it's imported from an external library. The presence of strcmp tells us the decoded password is compared to user input using a standard string comparison. This is our breakpoint target in a dynamic analysis scenario.


3.2 Disassembling decode_password

objdump -d --no-show-raw-insn bypassme.bin | grep -A 40 '<_Z15decode_passwordPc>'
Enter fullscreen mode Exit fullscreen mode

Breaking Down the Command

Component Meaning
objdump Disassembler for binary files
-d Disassemble executable sections
--no-show-raw-insn Hides raw hex bytes — cleaner output for reading
grep -A 40 Show 40 lines after the matching line
'<_Z15decode_passwordPc>' Match the specific function label

Why objdump and not ghidra or IDA Pro? Those tools are excellent but require a GUI or installation. objdump is available on virtually every Linux system by default, making it the universal first choice for quick static analysis in a remote shell environment.


3.3 Decoding the Password — The Core Vulnerability

From the disassembly of decode_password, we extracted this critical logic:

movabs $0xc9cff9d8cfdadff9,%rax    ; Load 8 encoded bytes into rax
mov    %rax,-0x13(%rbp)             ; Store on stack
movw   $0xd8df,-0xb(%rbp)          ; Load 2 more encoded bytes
movb   $0xcf,-0x9(%rbp)            ; Load 1 more encoded byte
Enter fullscreen mode Exit fullscreen mode

This gives us 11 encoded bytes stored on the stack:

f9 df da cf d8 f9 cf c9 df d8 cf
Enter fullscreen mode Exit fullscreen mode

⚠️ Important — Endianness: The value 0xc9cff9d8cfdadff9 is stored in little-endian format. This means the bytes are stored in reverse order in memory. So in memory, the sequence starts with f9, then df, then da, etc.

Then the decoding loop:

movzbl -0x13(%rbp,%rax,1),%eax   ; Load encoded byte[i]
xor    $0xffffffaa,%eax            ; XOR with 0xAA
mov    %dl,(%rax)                  ; Store decoded byte to output buffer
Enter fullscreen mode Exit fullscreen mode

The algorithm in plain English:

For each byte in the encoded array, XOR it with 0xAA. The result is the decoded password character.

Performing the XOR manually:

Index Encoded (hex) XOR 0xAA Decoded (ASCII)
0 0xF9 0xF9 ^ 0xAA 0x53 = S
1 0xDF 0xDF ^ 0xAA 0x75 = u
2 0xDA 0xDA ^ 0xAA 0x70 = p
3 0xCF 0xCF ^ 0xAA 0x65 = e
4 0xD8 0xD8 ^ 0xAA 0x72 = r
5 0xF9 0xF9 ^ 0xAA 0x53 = S
6 0xCF 0xCF ^ 0xAA 0x65 = e
7 0xC9 0xC9 ^ 0xAA 0x63 = c
8 0xDF 0xDF ^ 0xAA 0x75 = u
9 0xD8 0xD8 ^ 0xAA 0x72 = r
10 0xCF 0xCF ^ 0xAA 0x65 = e

Decoded password: SuperSecure

💡 Why XOR? XOR is the most common obfuscation technique in malware and CTF challenges because it is:

  • Reversible: XOR is its own inverse. A XOR key = B and B XOR key = A
  • Simple to implement: One assembly instruction
  • Easily broken: If you know the key (or can find it in the binary), it provides zero cryptographic security

3.4 Confirming the Authentication Flow in main

objdump -d --no-show-raw-insn bypassme.bin | grep -A 120 '<main>'
Enter fullscreen mode Exit fullscreen mode

This confirmed the exact execution path:

main:
  1. decode_password() → decoded password stored at rbp-0x110
  2. intro_sequence()  → shows the banner/animation
  3. fgets()           → reads user input into rbp-0x210
  4. sanitize()        → cleans user input
  5. strcmp(user_input, decoded_password) at 0x1759
  6. If equal → auth_sequence() → fopen("flag") → print flag
  7. If not equal → "Wrong password" → loop back (max 3 tries)
Enter fullscreen mode Exit fullscreen mode

The road not taken — Dynamic Debugging with LLDB:

The challenge hint specifically mentioned LLDB (a debugger). Here's how we would have used it if static analysis had failed:

# We would have set a breakpoint at the strcmp call address
lldb bypassme.bin
(lldb) b *0x1759          # Break at strcmp
(lldb) run
(lldb) x/s $rsi           # Read the decoded password from the rsi register
Enter fullscreen mode Exit fullscreen mode

In x86-64 Linux calling convention:

  • rdi = first argument to strcmp (user input)
  • rsi = second argument to strcmp (decoded password)

We chose not to use LLDB because static analysis gave us a complete answer with less complexity. Dynamic debugging introduces additional challenges (ASLR, needing to interact with the program, etc.). Always exhaust static analysis before resorting to dynamic analysis.


3.5 Obtaining the Flag

With the password SuperSecure recovered, we ran the binary and entered it:

./bypassme.bin
# At the prompt: SuperSecure
Enter fullscreen mode Exit fullscreen mode

The binary authenticated successfully, opened the flag file via fopen, and printed the flag.


4. "Privilege Escalation" — The SUID Mechanism

In this challenge, there was no traditional privilege escalation step because the SUID binary is the privilege escalation mechanism. Here's how it works:

┌─────────────────────────────────────────────────────┐
│  ctf-player (low privilege user)                    │
│  → runs bypassme.bin                               │
│  → OS sees SUID bit + owner = root                 │
│  → process runs as root                            │
│  → fopen("/path/to/flag") succeeds                 │
│  → flag contents printed to our terminal           │
└─────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

The flag file is readable only by root. Without the SUID binary, we could never access it. The binary acts as a controlled gateway — but since we bypassed the authentication check, we walked right through it.


5. Lessons Learned & Mitigation

5.1 Key Takeaways for Attackers (CTF Players)

Lesson Detail
Read symbols first nm and objdump reveal the entire structure of a binary in seconds
Not stripped = gift Debug symbols make reverse engineering dramatically faster
XOR is not encryption Single-byte XOR obfuscation is defeated trivially once the key is in the binary
Static before dynamic Exhaust static analysis before firing up a debugger
SUID = read the flag SUID root binaries are the key that unlocks root-owned files
Follow the strcmp In authentication binaries, find the comparison function and read its arguments

5.2 Mitigation Strategies for Defenders (Blue Team)

Vulnerability 1: Trivial XOR Obfuscation

What the developer did: Stored a password encoded with a single-byte XOR key (0xAA) — both the encoded bytes and the key are in the binary.

What should be done instead:

  • Never store passwords in a binary at all, even obfuscated
  • Use challenge-response authentication where the password never leaves the server
  • If a local comparison is unavoidable, use a proper cryptographic hash (bcrypt, Argon2) — an attacker who recovers a hash cannot reverse it
  • Use remote authentication — have the binary send the input to a server for verification; the "correct answer" never exists locally

Vulnerability 2: Binary Not Stripped / Debug Info Present

What the developer did: Compiled and deployed the binary with full debug symbols and without stripping.

What should be done instead:

  • Always strip production binaries: strip --strip-all bypassme.bin
  • Remove debug info at compile time (avoid -g flag in gcc/g++)
  • Use obfuscation tools like LLVM-Obfuscator for additional protection (note: this raises the bar, it doesn't solve the fundamental problem)

Detection (Blue Team Monitoring):

Alert: Process bypassme.bin executed by user ctf-player
  → Running with effective UID 0 (root)
  → Opened file: /root/flag.txt
  → This access pattern is anomalous for this user
Enter fullscreen mode Exit fullscreen mode

A proper SIEM or EDR (like Wazuh, Falco, or CrowdStrike) would flag a low-privilege user triggering a SUID binary that subsequently reads sensitive root-owned files.


Appendix: Full Attack Chain Summary

1. SSH into target
        ↓
2. ls -la → Discover SUID binary bypassme.bin
        ↓
3. file bypassme.bin → 64-bit ELF, not stripped, debug_info present
        ↓
4. nm bypassme.bin → Discover decode_password(), strcmp usage
        ↓
5. objdump disassembly of decode_password()
        ↓
6. Extract encoded bytes: f9 df da cf d8 f9 cf c9 df d8 cf
        ↓
7. XOR each byte with 0xAA → "SuperSecure"
        ↓
8. Run binary → Enter "SuperSecure" → FLAG OBTAINED
Enter fullscreen mode Exit fullscreen mode

Total tools used: ssh, ls, file, nm, objdump
No exploits. No brute force. No debugger needed.
Pure static analysis.


🎓 Final Thought for Beginners: This challenge perfectly illustrates why security-through-obscurity fails. The developer thought they were hiding the password by encoding it. But encoding is not encrypting. The moment we could read the binary's assembly, the "hidden" password was completely visible. Real security means an attacker can have full access to your code and still cannot compromise your system. That's the standard to aim for.

Top comments (0)