0x10Work / Vulnerability research
A stack overflow that never reaches RIP
Root-causing a crash in a vulnerable C server under GDB, and finding it does not fail the way the textbook says.
The setup
A deliberately vulnerable C TCP server from a book chapter on taint analysis. It reads from a socket in a loop and copies each chunk into a fixed stack buffer with memcpy. I reproduced the chapter, then went three steps further: measure the crash, read the stack frame live, and compare the same bug in Rust.
The crash matrix
| Build | Payload | Result |
|---|---|---|
-fno-stack-protector | 1024 B | SIGSEGV, exit 139 |
-fstack-protector | 1024 B | SIGABRT, exit 134, "stack smashing detected" |
-fno-stack-protector | 264 B | smallest crash (SIGSEGV 9 of 10 runs, SIGBUS 1 of 10) |
-fstack-protector | 137 B | smallest canary trip. 136 B survives |
Rust, [u8; 128] | oversized | panic, "range end index 256 out of range for slice of length 128", exit 101 |
Rust, Vec<u8> | oversized | survives |
| ASan | oversized | stack-buffer-overflow on finalBuffer at offset 320, caught at the first out-of-bounds byte |
What actually happens
I read both stack frames in GDB rather than inferring them. In the unprotected build, finalBuffer sits at rbp-272, buffer at rbp-144, bytesRead at rbp-16 and offset at rbp-4. Saved RIP is at +280 from the start of finalBuffer.
The overflow reaches the loop's own offset variable at +268, before it reaches saved RIP at +280. So the next offset += bytesRead runs on corrupted values: 0x41414141 + 0x4141414141414141, truncated to int, gives 0x82828282. cdqe sign-extends it to a large negative number, and the next memcpy writes about 2 GiB below the stack and faults.
RIP is overwritten. Control never reaches the ret. This is a crash with strong evidence of RIP control, not a demonstration of it.
In the protected build, GCC moves both character arrays next to the canary and pushes the scalars below them. The canary I measured was 0xefa877edee096700. The zero low byte marks glibc's terminator-style canary.
Why _FORTIFY_SOURCE cannot see it
Each memcpy is in bounds on its own. The violation only exists across loop iterations. I compiled with -D_FORTIFY_SOURCE=2 -O2, confirmed no __memcpy_chk was emitted, and the fortified build still crashed.
Checking my own claims
I wrote verify_all.sh: 32 checks (22 against GCC builds, 7 under GDB, 3 against rustc) that re-derive every claim in the write-up from the binaries and live runs. It trusts none of the saved logs, and its first check is the SHA-256 of the source listing. Running it falsified three of my own assertions, and I corrected the write-up. A separate ledger marks each of 20 claims as measured or inferred.
The static side maps sources, propagators and sinks onto CWE-787, CWE-121, CWE-193 and CWE-200, with an annotated disassembly of handleClient showing no bounds comparison before the copy. The Rust build has three unsigned comparisons and a cold slice_index_fail path in the same place.
Limits
No exploit. I stopped at root cause. The Windows port was not run, so nothing here covers /GS or WinDbg.