I am a student in networking, we learn cybersecurity, but we don't learn buffer overflow vulnerabilities in depth, I understand how it works, but I don't understand how an hacker can detect this vulnerability. Is that very difficult or can be done easily with some techniques ?

  • Look up fuzzing. That's one way.
    – nobody
    Feb 15, 2021 at 18:58

1 Answer 1


There are a number of ways. Probably the most common ones, though are fuzzing and reverse engineering.

Fuzzing is a form of automated security testing where a program that consumes untrusted input (such as a file parser, driver that accepts IOCTLs from less-privileged code, network server, or network client that connects to untrusted servers) is repeatedly subjected to algorithmically-generated inputs. There are several variants of fuzzing, such as mutational fuzzing (where valid inputs are mutated randomly before testing) and generational fuzzing (where the fuzzer is given a description of the expected data format and then generates inputs that approximate, though do not necessarily follow, that format). Fuzzing can also be anywhere from completely random changes to fully deterministic for a given data specification, though most fuzzers are somewhere in between (they use some logic to determine interesting locations within the input, and interesting values to change them to, but the actual details are randomized). This randomness means that fuzzing usually can't exhaustively test a program, but it's a highly effective way to find vulnerabilities nonetheless.

The program being fuzzed is run under a debugger that will detect when, where, and why it crashes (or exhibits other undesirable behavior, such as becoming unresponsive or using exorbitant memory). Even without debug symbols (much less source code), it is usually possible to determine what kind of exception the program encountered, which security researchers can use to estimate the likelihood of exploitability (for example, an NX violation where the process tries to execute a non-executable address, or a write access violation / segmentation fault where the program tries to write to a read-only or unmapped address, are both likely to be exploitable). In every case of a crash or other indication that the program failed to handle the input, the fuzzer logs the crash dump, type of failure, and specific input that caused the issue. However, it's highly unlikely that a fuzzing input will be a useful attack by itself (except possibly to crash the program for denial-of-service), so weaponizing the vulnerability (Exploit Development)usually takes more work.

Reverse Engineering is the process of figuring out the details of something's design, without the actual designs. In the context of software specifically, it often involves disassembling binary code into human-readable assembly language, and possibly "decompiling" to produce a higher-level programming language that is easier to understand and navigate. Neither disassembly nor decompilation can reliably generate the original source code. For example, things like comments are inevitably lost, many languages do not maintain data type info in their compiled form, optimizations can rearrange code and collapse constants and so on in ways that are harder for humans to read, and there are simply multiple ways to express some programming concepts (for and while loops are logically interchangeable even though they look a bit different). However, there exist tools that do a decent job with most programs, and some people are skilled at reading the output of such tools much the way they would read actual human-written source code. (Also, some languages are extremely easy to decompile, although those languages usually include built-in protections against buffer overflows in particular.)

Exploit development without source code usually requires some amount of very focused reverse engineering, where the vulnerable code is examined in detail to determine what input will have the desired result. This is often a somewhat interactive process, involving running the program many times under a debugger, single-stepping through the vulnerable code to see exactly what the registers and memory layout look like, and then altering the input to make the vulnerable program do the desired thing (typically loading a "shellcode" or similar, to enable executing an arbitrary program). This process is the same regardless of whether the vulnerability was found via fuzzing, reverse engineering of the whole binary, or something else.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .