Acfirth

Binary Exploitation - Part 1: Terminology

by acfirth

Date: 12 May 2025
Categories: BinaryExploitation

Introduction:

I have recently become fascinated with Binary Exploitation. The process of disassembling/decompiling binaries, reading through the Source Code or Assembly, and understanding where vulnerabilities lie within the binary as well as why the vulnerabilities exist, is intriguing to me.

I’ve found that when it comes to learning about Binary Exploitation, you need to ask yourself “but why?” at many points. You can’t just say “Oh yeah, that’s a vulnerability”, you need to understand why it’s a vulnerability. What programming or compiling error led to this vulnerability? What actually happens, in the code, that makes it exploitable? Why is it possible to exploit it?

You have to, somewhat, put yourself in the perspective of the developer and think why they chose to write the code in that specific way.

As of the time of me writing this, I am in no way an expert at Binary Exploitation, but I have decided to start this series of posts on the topic to not only document my learning experience, but to hopefully explain things in a manner that others, who want to understand more on the topic, can relate to and help them understand things a bit better.

Common Terminology:

When it comes to Binary Exploitation, you will come across many acronyms, phrases, and terms that seem mind-boggling at first. But once you understand what the terms mean, the tasks at hand tend to become much simpler.

x86, x86-64, and x64

x86

x86-64

x64

Buffer Overflow

A vulnerability which allows attackers to write more data into a fixed-size buffer than it can hold, possibly overwriting adjacent memory locations leading to data corruption and sometimes Arbitrary Code Execution.

Stack Smashing

A type of Buffer Overflow where an attacker can overwrite memory locations on the stack. Often leading to overwriting return addresses leading to a program crash or Arbitrary Code Execution.

ROP (Return-Oriented Programming) and ROP Chains

A method used when exploiting Buffer Overflows which use small snippets of Assembly instructions, already within the program’s memory (named gadgets), which can force the program to perform unintended actions. It can also be used to bypass security protections in place in the binary, such as a non-executable stack.

A ROP chain is simply multiple snippets of Assembly instructions chained together to get a specific outcome.

Shellcode

Shellcode is a (usually) small string of Assembly instructions, that is placed on the stack and then “jumped to” to execute it. Shellcode can be crafted to perform a multitude of operations, including (but not limited to):

Basically, if you can think of something you want to do on the target system. You can write some shellcode to do it. (With some limitations of course).

NOP (No Operation) Sled

A NOP Sled is a string of No Operation instructions that are commonly used as padding to prepend shellcode. When a NOP instruction is met in the execution of the binary, it simply does nothing. It meets the instruction and just goes “Okay, next instruction then”.

Often, pading is required when performing a Buffer Overflow to align the shellcode on the stack to make it easier to jump to. When using a NOP Sled, you do not need to be so precise with padding to get a perfect alignment. If you can write a long enough NOP Sled on the stack, you can make the execution of the binary jump to any location in the NOPs and it will simply travel all the way down the NOP Sled until the shellcode, where it will then be executed.

Format String Vulnerability

A format string vulnerability is when a function like printf() is used to output user controlled input or data. However, if the use of printf() is not programmed correctly to specify the output type (such as “string”) or the user controlled input or data is not sanitised, it can be abused to leak memory from the stack or even overwrite memory locations on the stack.

Overwriting memory locations is particularly dangerous as it could allow an attacker to overwrite a return address and cause the program to perform unintended actions or execute arbitrary code.

Heap Overflow

Similar to a Buffer Overflow, however this one affects the heap. It can allow an attacker to overwrite function pointers or return addresses leading to unintended actions or Arbitrary Code Execution.

Use After Free (UAF)

A Use After Free vulnerability is commonly a heap exploitation method which occurs when a program continues to use a pointer to a memory location after it has been freed.

This means that an attacker could overwrite the memory location with some malicious data after it has been freed, so when the program uses the pointer again it will point to the malicious data and execute it.

ASLR (Address Space Layout Randomization)

A security technique implemented to randomize the location of data structures in memory. Such as the:

It is used to make it harder for attackers to predict memory addresses and perform exploits.

DEP (Data Execution Prevention)

A security feature in Operating Systems that prevents code from being executed in certain areas of memory, such as the stack or the heap, making Buffer Overflows more difficult to perform.

Stack Canary

A security technique used to help prevent stack Buffer Overflows. A unique value, the “canary” is placed on the stack next to the return address of the current function being executed. Before the function returns, the canary value is checked against the original value. If the values match, then no overflow has occurred and the function returns as intended. However, if the canary values do not match, that means an overflow has occurred and it has been overwritten. This causes the program to halt execution and exit to prevent the exploit from working.

Format String Vulnerabilities can be abused to leak stack canaries which can allow an attacker to perform a Buffer Overflow attack, overwriting the stack canary with its original value as to not trigger the security measure but still exploit the vulnerability.

Control Flow Hijacking

A method of exploiting a program by taking control of it’s execution flow. Common examples include overwriting function return addresses or manipulating the program’s jump table.

Ret2Win

A term really only used in challenge or CTF contexts. There tends to be a “win” function, that does something like printing a flag or spawning a shell. The aim is to find a way to overwrite the return address of a function with the address of the “win” function, as to “Return to Win”.

Segfault (Segmentation Fault)

A common error that occurs when a program attempts to access memory that it is not allowed to, often used as an indicator of buffer overflow or other memory corruption vulnerabilities.

Fuzzing

A technique used to automatically input different length strings of data or malformed data into a program to discover vulnerabilities such as Buffer Overflows.

Dynamic Analysis

The process of running a program in a controlled environment, such as a debugger like GDB, to view important feature like the stack and registers. This can be helpful when writing exploits to make sure your exploit is performing as intended.

Static Analysis

The process of analysing a binary without running it to locate vulnerbailities. This could be reading the source code or using a decompiler or disassembler to view the pseudocode or the Assembly.

GOT (Global Offset Table)

A table used by programs to store addresses of functions and variables from shared libraries. Exploiting the GOT can allow attackers to redirect the flow of the program by modifying the function pointers.

PLT (Procedure Linkage Table)

The PLT is a section in ELF (Unix and Linux) binaries that handles dynamic function resolution for shared libraries.

Final Words:

This is just a select few key words and phrases that you are very likely to come across in your Binary Exploitation learning journey. There are many more which I will discuss in later parts of this series. But for now, knowing what these key bits of terminology mean, will help you immensely.

In the next part of this series, I will discuss the registers in x86 and x64 binaries. How they differ and what they are used for.


Tags: Cybersecurity - Binary Exploitation - PWN - Buffer Overflow