How Exploits Work: Vulnerability Classes and Memory Safety Concepts
An exploit is a way of using a vulnerability to make a system do something its designers never intended. Understanding how exploits work, at the level of ideas rather than recipes, is essential for defenders. It explains why certain bugs are dangerous, why patches matter and why modern operating systems include protections such as ASLR and DEP. This lesson stays conceptual: you will learn the vulnerability classes, the defences built into modern systems and how safe coding removes whole categories of flaws.
Vulnerability, Exploit, Payload: Three Different Things
| Term | Meaning | Analogy |
|---|---|---|
| Vulnerability | The flaw itself | A weak lock on a door |
| Exploit | The method that takes advantage of the flaw | The technique used to defeat the lock |
| Payload | What the attacker wants to do once the flaw is used | What the intruder does once inside |
Separating these ideas matters when you write reports: the fix addresses the vulnerability, while detection often focuses on the exploit behaviour and the payload.
Common Vulnerability Classes
| Class | What goes wrong | Typical defence |
|---|---|---|
| Buffer overflow | A program writes more data into a memory area than it can hold, overwriting neighbouring memory | Bounds checking, memory-safe languages, compiler protections |
| Use after free | A program keeps using memory after releasing it, so the data there may be controlled by someone else | Ownership models, safer allocators, memory-safe languages |
| Integer overflow | Arithmetic wraps around, producing a small size where a large one was expected | Checked arithmetic, input validation |
| Format string flaw | User input is used as a formatting template | Always use fixed format strings |
| Race condition (TOCTOU) | State changes between a check and its use | Atomic operations, locking, avoiding check-then-act patterns |
| Injection | Data is interpreted as commands (SQL, shell, template) | Parameterisation, input validation, least privilege |
| Deserialisation flaw | Untrusted data rebuilds objects that trigger code paths | Avoid native deserialisation of untrusted data, use simple formats, signatures |
| Logic flaw | The design lets users bypass intended rules | Threat modelling, abuse-case testing, code review |
The Idea Behind Memory Corruption
Languages such as C and C++ give programmers direct control of memory but do not check that they stay inside the bounds. Picture a row of lockers, each a fixed size. If a program tries to put a long item into a short locker and nobody checks the length, the item spills into the next locker and overwrites its contents. If the neighbouring contents include something the program trusts (a saved return address, a function pointer, a security flag), the program’s behaviour can change. That is the essence of memory corruption. You do not need working exploit code to understand why this is serious: the program’s control flow depends on data an outsider influenced.
Unsafe versus safe code: a C example
Here is the classic flaw and its fix. This is a teaching illustration of a coding mistake, not an attack.
/* UNSAFE: no check that name fits in buffer */
void greet_unsafe(const char *name) {
char buf[16];
strcpy(buf, name); /* copies until the end of name, however long */
printf("Hello, %s\n", buf);
}
/* SAFER: size-limited copy and guaranteed termination */
void greet_safe(const char *name) {
char buf[16];
snprintf(buf, sizeof(buf), "%s", name); /* never writes past 16 bytes */
printf("Hello, %s\n", buf);
}
The first function trusts the length of the input. The second states the limit explicitly. Code review tools, compiler warnings and static analysers flag functions such as strcpy, sprintf and gets for this reason.
Modern Protections Built Into Systems
Decades of memory-corruption bugs led to layered defences. None is perfect, but together they make exploitation much harder and more expensive.
| Protection | What it does |
|---|---|
| DEP / NX (data execution prevention) | Marks data areas such as the stack as non-executable, so injected data cannot simply be run as code |
| ASLR (address space layout randomisation) | Places code and data at unpredictable memory addresses each time a program runs |
| Stack canaries | Places a secret value before a saved return address and checks it before returning; a changed value aborts the program |
| RELRO / PIE | Makes parts of the executable and its function tables harder to overwrite or predict |
| Control-flow integrity (CFI) and shadow stacks | Check that execution jumps only to legitimate targets |
| Sandboxing and privilege separation | Limit what a compromised process can do (browsers, mobile apps, containers) |
| Hardened allocators and memory tagging | Detect or prevent heap misuse such as use after free |
See the protections on your own lab machine
On a Linux lab VM, check whether ASLR is enabled and inspect the protections of a program you compiled yourself:
cat /proc/sys/kernel/randomize_va_space # 2 means full ASLR
# Compile a harmless test program with and without protections
gcc -o demo_safe demo.c # distributions enable many protections by default
gcc -fno-stack-protector -o demo_plain demo.c
file demo_safe demo_plain
checksec --file=demo_safe # if the checksec tool is installed
Compare the two outputs. The point of the exercise is to understand which mitigations exist, not to defeat them.
Memory-Safe Languages and Secure Defaults
The most effective fix for memory-corruption bugs is to avoid writing them. Languages such as Rust, Go, Java, C#, Python and Swift check bounds and manage memory for the programmer, which removes most buffer overflows and use-after-free errors. Government agencies and large vendors now openly recommend adopting memory-safe languages for new projects and migrating the riskiest components. Where C or C++ must be used, adopt:
- Safe library functions and containers with bounds checking.
- Compiler warnings treated as errors (
-Wall -Wextra -Werror) and hardening flags. - Static analysis and fuzzing, which feeds many random inputs to find crashes.
- Sanitisers during testing (AddressSanitizer, UndefinedBehaviorSanitizer).
- Code review focused on input handling and memory ownership.
Practice Lab: Find the Bug (Defender’s Version)
- Compile the unsafe and safe functions above into a small test program in your lab VM. Run it with short names, then with a name longer than 16 characters.
- Compile with
-fsanitize=address -gand rerun. AddressSanitizer will report the problem and show where the bad write happened. Read the report line by line. - Fix the unsafe function using
snprintfand confirm the report disappears. - Try a free-memory mistake: allocate with
malloc, callfree, then print the pointer’s data. Run it with the sanitiser and read how the tool describes a use-after-free. - Rewrite the same small program in Python or Rust and note which mistakes the language prevents automatically.
How Defenders Detect and Respond
- Patch quickly when a memory-corruption vulnerability appears in widely used software, since public exploit knowledge tends to follow disclosure.
- Use endpoint protection capable of detecting exploit behaviour such as unusual process injection and unexpected child processes.
- Reduce attack surface: disable unneeded services and features.
- Monitor crashes: repeated crashes of a service can indicate someone probing it.
- Segment and sandbox so that a compromised application cannot easily reach everything else.
Practice Exercises
- Explain in your own words the difference between a vulnerability, an exploit and a payload.
- Why does ASLR make exploitation harder even when a vulnerability exists? Why is it not a complete fix?
- Find three unsafe C functions and write the safer alternative for each.
- List three vulnerability classes that memory-safe languages mostly eliminate, and three they do not.
- Draw a diagram of a stack frame with a canary and explain what happens if the canary value changes.
Mentor tip: You do not need to be an exploit developer to be an excellent security professional, but you do need to understand why classes of bugs exist. That understanding lets you ask better questions during code review, risk assessment and incident response.
Educational use only: this lesson explains concepts for defence. Test only software you own or are authorised to assess.
- A vulnerability is the flaw, an exploit is the technique and a payload is the goal.
- Memory corruption bugs occur when programs fail to check bounds or manage memory correctly.
- DEP, ASLR, canaries and CFI raise the cost of exploitation but do not remove the bugs.
- Memory-safe languages, sanitisers, fuzzing and code review prevent whole bug classes.
- Patch promptly and reduce attack surface to limit exposure.