Can some one give me a start point for finding exploits in software and reverse engineering?

  • I need some kind of a book or a site...
  • I have background with programming with high and low level such as assembly...
  • I want to know how the "hackers" find all the exploits on OS's... For example I'v heard that some one found an exploits on apache that is running on linux using reverse engineering... .

First of all, in order to be able to perform reverse engineering you should build some good C programming skills and be able to understand some Assembly commands too.

The "problem" with reversing engineering is that you lose a lot of code information depending on the disassemblers or decompilers used (e.g.: code comments, variable name, data types and so on...). That makes your life as vulnerability researcher complicated.

I suggest as a starting point to take a look at C programs. Look for user supplied data commands such as: argv, gentenv(), read(), getc(), scanf() and recv() to check if the programmer has protected the inputs handling against buffer overflows.

There are good freeware tools in market which make your life a bit easier. Take a look at this book: ISBN 978-0-07-183238-0. They have listed some of them.


There are typically three things you can do when searching for buffer overflows:

Source Code Review

If the source code is available, you could possibly find it by reviewing the code.

Decompile the code and review the source

If the source code is not available, reverse engineering could be one of the options. Once discovered it is possible to do so, the source code can be reviewed.

Fuzzing and Debugging

If the first two options are not possible due to code obfuscation or any other method of protecting binaries, fuzzing and debugging is an option.

This basically means that by sending a large buffer to a specific input field and see how the application handles it.

A good resource for Windows 32 bit application to start with is: resources.infosecinstitute.com/stack-based-buffer-overflow-in-win-32-platform-part-1/

Keep in mind that memory protections such as DEP and ASLR could cause issues while exploiting a program or service.

  • Can you name the other methods which make source code review and decompile impossible? Thanks!
    – user69377
    Sep 2 '15 at 5:23
  • 2
    It's never impossible as it's always a matter of time and the amount of effort you want to put into it. Source code obfuscation tools make it harder to read the code after decompilation. Crypters make it a lot harder to decompile the code. Dotfuscator is an example of a .NET obfuscator (preemptive.com) and an example of a crypter is CypherX (cypherx.org). Don't be confused when it comes to crypters, often they're used to bypass anti-virus checks but that's not the only purpose.
    – Jeroen
    Sep 2 '15 at 5:43

The best resources out there today are 2 books:

The Art of Software Assessment, Dowd, McDonald and Schuh. A work of art. This book will teach you everything you need to know about the methodology, threat analysis and how exploits work and what to look for.

The Tangled Web, Michal Zalweski. The best book out there right now for securing Web applications. Written by a master.

Not the answer you're looking for? Browse other questions tagged or ask your own question.