The Stack...

In the x86 primer, the stack got one chapter. That turned out to be far too little for the one piece of memory that every function touches, every hook depends on, and most crashes eventually trace back to. Most explanations stop at "it grows downward, use push and pop". That is true, and it is also exactly where the interesting questions start: Why does nearly every function begin with sub rsp, 28h? Why does a harmless-looking movaps crash your hook? What is actually sitting at [rsp+30h]? And what really happens when the stack "overflows"?

This post is a dedicated deep dive. We will look at what the stack physically is, how a Windows x64 stack frame is laid out, why alignment matters, how to save and restore state safely in your own injected code, and what goes wrong when the stack breaks. The focus is Windows x64, because that is where nearly every game you will ever attach a debugger to runs.

If you can look at [rsp+38h] inside a function and say whether it is a local variable, a saved register, the return address, or an argument from the caller, the stack has stopped being magic.

What the stack actually is#

The stack is ordinary memory. There is no special hardware behind it. What makes it special is how it is used: one register, RSP, marks the boundary between used and free space, and a handful of instructions (push, pop, call, ret) move that boundary implicitly.

• One stack per thread » When the OS creates a thread, it reserves a contiguous block of memory for that thread's stack. Two threads never share a stack, which is why local variables are naturally private to the thread that owns them.
• Fixed maximum size » On Windows, a thread reserves 1 MB by default. The value comes from the executable's PE header and can be changed by the linker or per thread. On Linux, 8 MB is a typical default for the main thread (see ulimit -s).
• Grows downward » New data goes to lower addresses. Reserving space makes RSP smaller, releasing space makes it larger.

             higher addresses
  +---------------------------+ <- StackBase (where the stack starts)
  | frames of earlier callers |
  | ...                       |
  | caller's frame            |
  | current function's frame  |
  +---------------------------+ <- RSP  (the "top" of the stack)
  | free                      |    |
  | ...                       |    v  grows this way
  +---------------------------+ <- StackLimit
  | guard page                |
  +---------------------------+
             lower addresses

Keep one rule from this diagram: everything at or above RSP is in use, everything below RSP is free. The "top" of the stack is its lowest used address, which is why the vocabulary feels upside down at first.

Where is it? Asking the TEB#

On Windows x64, the GS segment points to the current thread's TEB (Thread Environment Block), and its first fields describe the stack:

mov rax, gs:[8]      ; StackBase  – upper end, where this thread's stack starts
mov rcx, gs:[10h]    ; StackLimit – current lower end (moves down as the stack grows)

RSP always lies between these two values. That makes a handy sanity check while debugging: if a pointer falls between StackLimit and StackBase, it points into this thread's stack.

Reserving and releasing space#

Every use of the stack follows the same pattern: move RSP down to reserve room, use the room, move RSP back up.

sub  rsp, 10h        ; reserve 16 bytes
mov  [rsp], rax      ; use them
mov  [rsp+8], rcx
; ...
add  rsp, 10h        ; release them

Two details are worth burning in early:

• sub rsp does not touch memory » It is plain arithmetic on a register. It allocates nothing in the malloc sense and cannot crash on its own. The memory already belongs to the thread's stack region; you are only moving the boundary between "in use" and "free".
• Releasing does not erase » After add rsp, 10h, the 16 bytes still hold the old values of RAX and RCX. They are simply considered free and will be overwritten by the next push, call, or function frame. This is why uninitialized local variables contain "garbage": it is whatever the previous function left behind.

push and pop are nothing more than shortcuts for this pattern with a size of 8 bytes, as covered in the primer: push rbx is sub rsp, 8 plus mov [rsp], rbx, and pop rbx is the reverse.

Offsets move when RSP moves#

[rsp+X] is not a fixed location. It is relative to wherever RSP points right now, and every push or sub rsp changes what a given offset refers to:

; RSP = 1000h, the value we want sits at [rsp+20h] = 1020h
push rax               ; RSP = 0FF8h
mov  rcx, [rsp+20h]    ; reads 1018h – wrong slot!
mov  rcx, [rsp+28h]    ; reads 1020h – correct: +8 for the push

Compilers track this automatically. Humans writing hooks do not, and this is one of the most common silent bugs: nothing crashes, you just read the wrong value. The rule is simple: after every push, add 8 to later RSP-relative offsets; after every sub rsp, N, add N.

This is also the reason debug builds like RBP as a frame pointer. After push rbp and mov rbp, rsp, the address in RBP stays fixed for the whole function, so [rbp-8] is the same slot no matter how often RSP moves afterwards.

CALL, RET, and the return address#

call pushes the address of the next instruction and jumps; ret pops that address into RIP. The primer covered the mechanics. Here are the consequences that matter in practice:

• At a function's first instruction, [rsp] is the return address » Always, without exception. If you break on the first instruction of a function and read [rsp], you know exactly where it was called from: the instruction right before that address is the call.
• Call stacks are reconstructed from return addresses » When a debugger shows you a call stack, it is walking up through the frames and collecting return addresses. On Windows x64 it does not rely on an RBP chain for this; it uses unwind tables stored in the executable that describe each function's frame size.
• The call itself shifts alignment by 8 » The pushed return address moves RSP by 8 bytes. This small detail is the reason behind half of the "magic numbers" in prologues, as we will see in the alignment section.

Anatomy of a Windows x64 frame#

Suppose function A calls function B. This is what the stack looks like while B runs:

               higher addresses
  |  ... A's locals ...          |
  |  argument 6                  |  [rsp+30h]  -+
  |  argument 5                  |  [rsp+28h]   |
  |  shadow slot for R9          |  [rsp+20h]   |  offsets as seen
  |  shadow slot for R8          |  [rsp+18h]   |  at B's first
  |  shadow slot for RDX         |  [rsp+10h]   |  instruction
  |  shadow slot for RCX         |  [rsp+8]     |
  |  return address into A       |  [rsp]      -+
  +------------------------------+
  |  B's saved registers         |
  |  B's locals / spill slots    |
  |  B's outgoing shadow space   |  (for calls B makes)
  +------------------------------+ <- RSP while B's body runs
               lower addresses

Every frame has the same four zones: the return address (pushed by call), saved registers, locals and spill slots, and an outgoing area for the calls this function makes itself.

Arguments: registers first, stack second#

The first four arguments travel in RCX, RDX, R8, R9 (or XMM0–XMM3 for floats). Argument five onward goes on the stack, and its offset depends on where you are standing:

The difference of 8 is the return address that call pushed in between.

Shadow space: 32 bytes the caller always pays for#

Before every call, the caller reserves 32 bytes (0x20) directly above the return address, one 8-byte slot for each register argument. This happens even if the callee takes fewer than four arguments, or none at all. The space belongs to the callee, which may use it for anything. Debug builds typically copy the register arguments there at function entry (mov [rsp+8], rcx). Optimized MSVC code very often uses it to save non-volatile registers without a push:

my_function:
  mov  [rsp+8], rbx      ; save RBX in the caller's shadow slot for RCX
  mov  [rsp+10h], rsi    ; save RSI in the slot for RDX
  push rdi               ; one real push
  sub  rsp, 20h          ; shadow space for the calls my_function makes
  ; ...
  mov  rbx, [rsp+30h]    ; restore RBX: 8 + 8 (push) + 20h (sub) = 30h
  mov  rsi, [rsp+38h]    ; restore RSI: 10h + 28h = 38h
  add  rsp, 20h
  pop  rdi
  ret

Reading a prologue to map the frame#

The prologue tells you everything you need to name every slot. After the three prologue instructions above, RSP sits 0x28 bytes below where it was at entry, and the frame looks like this:

The method is always the same: start at the return address (it sits at "everything the prologue subtracted"), count the pushes and the sub back down, and count 8-byte slots upward into the caller's area. Once you can build this table from a prologue, every [rsp+X] in the function body has a name.

Alignment: the 16-byte rule#

The Windows x64 convention requires RSP to be a multiple of 16 at every call instruction. Since call then pushes 8 bytes, every function starts with RSP exactly 8 bytes off. Its prologue has to fix that before the body runs.

Why bother? Because compilers rely on it. They place 16-byte values on the stack and access them with aligned SSE instructions such as movaps and movdqa. Those instructions fault on an address that is not divisible by 16, and Windows reports that fault as an access violation.

The arithmetic is simple: at entry, RSP is 8 off. Each push adds 8, the sub adds its value. The body is aligned when 8 + 8 × pushes + sub is a multiple of 16 (0x10):

That solves the mystery from the introduction: sub rsp, 28h is 0x20 of shadow space for outgoing calls plus 8 bytes to undo the misalignment caused by the return address.

Using the stack in your own hooks#

When you inject code, it runs in the middle of someone else's function, on their stack, with their RSP. The compiler that built that function made assumptions about every register and every stack slot, and your code has to keep all of them intact. Three rules cover almost everything:

• Leave RSP where you found it » When you jump back to the original code, RSP must be exactly where it was when your hook started.
• Restore everything you change » Registers, flags, and XMM registers alike.
• Respect the frame » Read above RSP as much as you like, but write there only if you mean to change the function's data. And never store anything below RSP without moving RSP first.

General-purpose registers: push and pop#

push rax
push rcx
; ... your code, free to use RAX and RCX
pop  rcx             ; reverse order!
pop  rax

Pops must mirror pushes in reverse order. If you get the order wrong, nothing crashes: RAX and RCX simply swap values, and the bug surfaces somewhere else entirely.

Flags: pushfq and popfq#

Your cmp, test, add, or sub overwrites RFLAGS. If your hook sits between a cmp and its conditional jump, or the code after your hook reads flags for any other reason, you just changed the program's decision. Saving the flags is cheap, so when in doubt, do it:

pushfq
; ... code that changes flags
popfq

XMM registers: reserve and move#

There is no push for XMM registers. You reserve 16 bytes per register and store it yourself:

sub    rsp, 20h
movups [rsp], xmm0
movups [rsp+10h], xmm1
; ... your code
movups xmm1, [rsp+10h]
movups xmm0, [rsp]
add    rsp, 20h

Volatile and non-volatile registers#

The calling convention splits registers into two groups. A called function may destroy the volatile ones freely, but must restore the non-volatile ones before it returns:

This table matters the moment your hook calls a function: everything in the left column may come back changed.

Calling a function from a hook#

A call from injected code needs three things: the volatile registers saved, RSP aligned to 16 at the call, and 32 bytes of shadow space. Since you usually do not know the alignment at the hook location, the robust approach is to align explicitly and restore RSP from a saved copy:

pushfq
push rax
push rcx
push rdx
push r8
push r9
push r10
push r11

push rbp
mov  rbp, rsp          ; remember RSP
and  rsp, -10h         ; round RSP down to a multiple of 16
sub  rsp, 20h          ; shadow space for the callee

mov  rcx, rbx          ; example: first argument
call some_function

mov  rsp, rbp          ; undo the alignment and shadow space in one step
pop  rbp

pop  r11
pop  r10
pop  r9
pop  r8
pop  rdx
pop  rcx
pop  rax
popfq

and rsp, -10h clears the low four bits of RSP. Because the stack grows downward, rounding down only ever adds a few bytes of extra room and never overlaps data that is in use. If you need the function's return value, copy it out of RAX before the pops restore the old value. If the surrounding code has live floating-point values, back up XMM0–XMM5 as well, using the movups pattern from above.

When things go wrong#

Stack bugs have recognizable symptoms. Knowing them saves hours of staring at the wrong place.

Unbalanced stack: ret jumps into nonsense#

A push without its pop, or a sub rsp without its add, leaves RSP in the wrong place. The original function keeps running, and when it reaches ret, [rsp] is not the return address but one of your leftover values. RIP lands at an address that looks like data: a small integer, a float's bit pattern, a heap pointer. The crash happens far from your hook, which makes it confusing. When you see it, count pushes and pops along every path through your code, including early exits that jump back to the original code.

Misaligned stack: a crash on an innocent movaps#

Your hook works, but a function you call (or the code after your hook) crashes on a movaps with an access violation. Check the alignment at your call, and check whether you left RSP shifted by 8.

Wrong offset: no crash, just wrong data#

You read [rsp+20h] after a push and get a neighboring value. Nothing crashes, the cheat simply does not work. Recompute every RSP-relative offset after each push and sub.

Stack overflow: running out of room#

A stack overflow does not mean RSP drops below zero. RSP holds an address far away from zero. It means RSP has moved past the lower end of the region the OS reserved for the thread. On Windows, that region is reserved up front but committed gradually:

             higher addresses
  +---------------------------+ <- StackBase
  | committed, in use         |
  +---------------------------+ <- RSP
  | committed, free           |
  +---------------------------+
  | guard page                | <- touched: OS commits more, moves guard down
  +---------------------------+
  | reserved, not committed   |
  | ...                       |
  +---------------------------+ <- end of the reservation (e.g. 1 MB)
             lower addresses

When a stack access touches the guard page, the OS commits more memory and moves the guard page further down. Once the reservation is used up, there is nothing left to commit, and the thread gets STATUS_STACK_OVERFLOW (0xC00000FD). On Linux, the same situation ends in SIGSEGV. The usual causes are unbounded recursion and very large local arrays.

Note the trigger: it is a memory access, not the sub rsp itself. That creates a subtle problem for functions with large frames. If a function subtracts a lot from RSP at once and then touches the far end of its frame first, the access can land beyond the guard page, in memory that is reserved but not committed, and the result is an access violation instead of stack growth. For that reason, compilers insert a call to __chkstk into functions with large frames (Microsoft's documentation puts the threshold at 4 KB of locals for x86 and 8 KB for x64). __chkstk touches the new frame page by page, so the guard page mechanism works as intended.

Stack buffer overflow is something else#

The names are confusingly close. A stack overflow means running out of stack. A stack buffer overflow means writing past the end of a local buffer, into whatever lies above it: saved registers and, crucially, the return address. The next ret then jumps wherever the overwritten value points.

MSVC defends against this with security cookies (/GS): a random value placed between the locals and the return address, checked right before the function returns. If it changed, the process is terminated immediately with STATUS_STACK_BUFFER_OVERRUN (0xC0000409). Windows uses that same code for other fail-fast terminations, so it does not always mean a real buffer overrun, but it always means the process killed itself on purpose.

The stack in Cheat Engine#

Who called this function?#

Set a breakpoint on the first instruction of a function. When it hits, read [rsp]: that is the return address. Look at that address in the disassembler, and the instruction right before it is the call that got you here. Repeat the process in the caller to walk further up the chain.

Spotting return addresses in a stack dump#

While paused at a breakpoint, look at the memory starting at RSP. Most values will be data, but any value that points into a module's code (something like game.exe+1A2B3C) is almost certainly a return address. Those values are the breadcrumbs of the call chain.

Arguments inside a mid-function hook#

By the time execution reaches your hook, RCX, RDX, R8, and R9 may long since hold other values. The original arguments may have been copied to the shadow slots, moved into non-volatile registers, or stored in locals. Read the prologue (see mapping a frame) to find out where they went.

Stack addresses are not cheat entries#

As covered in the primer: a value found at [rsp+40h] lives in a frame that is rebuilt on every call and destroyed on every return. Its address is meaningless a moment later. Stable access goes through base pointers and offsets, not through the stack.

Cheat sheet#

Final thoughts#

The stack looks like a minor detail next to registers and instructions, but it is the backbone that holds everything together: every call, every return, every local variable, and every hook you write depends on RSP being exactly where the code expects it.

The good news is that the rules are few and strict. Once you track RSP as carefully as the compiler does, frames become readable, crashes become explainable, and your injected code stops being a gamble.