How Exploits Work: Vulnerability Classes and Memory Safety Concepts

An exploit is a way of using a vulnerability to make a system do something its designers never intended. Understanding how exploits work, at the level of ideas rather than recipes, is essential for defenders. It explains why certain bugs are dangerous, why patches matter and why modern operating systems include protections such as ASLR and DEP. This lesson stays conceptual: you will learn the vulnerability classes, the defences built into modern systems and how safe coding removes whole categories of flaws.

Vulnerability, Exploit, Payload: Three Different Things

TermMeaningAnalogy
VulnerabilityThe flaw itselfA weak lock on a door
ExploitThe method that takes advantage of the flawThe technique used to defeat the lock
PayloadWhat the attacker wants to do once the flaw is usedWhat the intruder does once inside

Separating these ideas matters when you write reports: the fix addresses the vulnerability, while detection often focuses on the exploit behaviour and the payload.

Common Vulnerability Classes

ClassWhat goes wrongTypical defence
Buffer overflowA program writes more data into a memory area than it can hold, overwriting neighbouring memoryBounds checking, memory-safe languages, compiler protections
Use after freeA program keeps using memory after releasing it, so the data there may be controlled by someone elseOwnership models, safer allocators, memory-safe languages
Integer overflowArithmetic wraps around, producing a small size where a large one was expectedChecked arithmetic, input validation
Format string flawUser input is used as a formatting templateAlways use fixed format strings
Race condition (TOCTOU)State changes between a check and its useAtomic operations, locking, avoiding check-then-act patterns
InjectionData is interpreted as commands (SQL, shell, template)Parameterisation, input validation, least privilege
Deserialisation flawUntrusted data rebuilds objects that trigger code pathsAvoid native deserialisation of untrusted data, use simple formats, signatures
Logic flawThe design lets users bypass intended rulesThreat modelling, abuse-case testing, code review

The Idea Behind Memory Corruption

Languages such as C and C++ give programmers direct control of memory but do not check that they stay inside the bounds. Picture a row of lockers, each a fixed size. If a program tries to put a long item into a short locker and nobody checks the length, the item spills into the next locker and overwrites its contents. If the neighbouring contents include something the program trusts (a saved return address, a function pointer, a security flag), the program’s behaviour can change. That is the essence of memory corruption. You do not need working exploit code to understand why this is serious: the program’s control flow depends on data an outsider influenced.

Unsafe versus safe code: a C example

Here is the classic flaw and its fix. This is a teaching illustration of a coding mistake, not an attack.

/* UNSAFE: no check that name fits in buffer */
void greet_unsafe(const char *name) {
    char buf[16];
    strcpy(buf, name);          /* copies until the end of name, however long */
    printf("Hello, %s\n", buf);
}

/* SAFER: size-limited copy and guaranteed termination */
void greet_safe(const char *name) {
    char buf[16];
    snprintf(buf, sizeof(buf), "%s", name);   /* never writes past 16 bytes */
    printf("Hello, %s\n", buf);
}

The first function trusts the length of the input. The second states the limit explicitly. Code review tools, compiler warnings and static analysers flag functions such as strcpy, sprintf and gets for this reason.

Modern Protections Built Into Systems

Decades of memory-corruption bugs led to layered defences. None is perfect, but together they make exploitation much harder and more expensive.

ProtectionWhat it does
DEP / NX (data execution prevention)Marks data areas such as the stack as non-executable, so injected data cannot simply be run as code
ASLR (address space layout randomisation)Places code and data at unpredictable memory addresses each time a program runs
Stack canariesPlaces a secret value before a saved return address and checks it before returning; a changed value aborts the program
RELRO / PIEMakes parts of the executable and its function tables harder to overwrite or predict
Control-flow integrity (CFI) and shadow stacksCheck that execution jumps only to legitimate targets
Sandboxing and privilege separationLimit what a compromised process can do (browsers, mobile apps, containers)
Hardened allocators and memory taggingDetect or prevent heap misuse such as use after free

See the protections on your own lab machine

On a Linux lab VM, check whether ASLR is enabled and inspect the protections of a program you compiled yourself:

cat /proc/sys/kernel/randomize_va_space     # 2 means full ASLR
# Compile a harmless test program with and without protections
gcc -o demo_safe demo.c                      # distributions enable many protections by default
gcc -fno-stack-protector -o demo_plain demo.c
file demo_safe demo_plain
checksec --file=demo_safe                    # if the checksec tool is installed

Compare the two outputs. The point of the exercise is to understand which mitigations exist, not to defeat them.

Memory-Safe Languages and Secure Defaults

The most effective fix for memory-corruption bugs is to avoid writing them. Languages such as Rust, Go, Java, C#, Python and Swift check bounds and manage memory for the programmer, which removes most buffer overflows and use-after-free errors. Government agencies and large vendors now openly recommend adopting memory-safe languages for new projects and migrating the riskiest components. Where C or C++ must be used, adopt:

  • Safe library functions and containers with bounds checking.
  • Compiler warnings treated as errors (-Wall -Wextra -Werror) and hardening flags.
  • Static analysis and fuzzing, which feeds many random inputs to find crashes.
  • Sanitisers during testing (AddressSanitizer, UndefinedBehaviorSanitizer).
  • Code review focused on input handling and memory ownership.

Practice Lab: Find the Bug (Defender’s Version)

  1. Compile the unsafe and safe functions above into a small test program in your lab VM. Run it with short names, then with a name longer than 16 characters.
  2. Compile with -fsanitize=address -g and rerun. AddressSanitizer will report the problem and show where the bad write happened. Read the report line by line.
  3. Fix the unsafe function using snprintf and confirm the report disappears.
  4. Try a free-memory mistake: allocate with malloc, call free, then print the pointer’s data. Run it with the sanitiser and read how the tool describes a use-after-free.
  5. Rewrite the same small program in Python or Rust and note which mistakes the language prevents automatically.

How Defenders Detect and Respond

  • Patch quickly when a memory-corruption vulnerability appears in widely used software, since public exploit knowledge tends to follow disclosure.
  • Use endpoint protection capable of detecting exploit behaviour such as unusual process injection and unexpected child processes.
  • Reduce attack surface: disable unneeded services and features.
  • Monitor crashes: repeated crashes of a service can indicate someone probing it.
  • Segment and sandbox so that a compromised application cannot easily reach everything else.

Practice Exercises

  1. Explain in your own words the difference between a vulnerability, an exploit and a payload.
  2. Why does ASLR make exploitation harder even when a vulnerability exists? Why is it not a complete fix?
  3. Find three unsafe C functions and write the safer alternative for each.
  4. List three vulnerability classes that memory-safe languages mostly eliminate, and three they do not.
  5. Draw a diagram of a stack frame with a canary and explain what happens if the canary value changes.

Mentor tip: You do not need to be an exploit developer to be an excellent security professional, but you do need to understand why classes of bugs exist. That understanding lets you ask better questions during code review, risk assessment and incident response.

Educational use only: this lesson explains concepts for defence. Test only software you own or are authorised to assess.

Key takeaways
  • A vulnerability is the flaw, an exploit is the technique and a payload is the goal.
  • Memory corruption bugs occur when programs fail to check bounds or manage memory correctly.
  • DEP, ASLR, canaries and CFI raise the cost of exploitation but do not remove the bugs.
  • Memory-safe languages, sanitisers, fuzzing and code review prevent whole bug classes.
  • Patch promptly and reduce attack surface to limit exposure.

Related Lessons