Take the 2-minute tour ×
Information Security Stack Exchange is a question and answer site for Information security professionals. It's 100% free, no registration required.

In software reverse engineering, we use a dissembler, a debugger, and a code patcher. OllyDbg includes all. One of the differences between actual execution of a program and debugging through using OllyDbg is, in OllyDbg the instruction pipe-lining is not used.

How can we take advantage of this fact to make the software immune to reverse engineering attacks?

The obvious answer is write an instruction that modifies already fetched instruction.

My question is how do we write a higher level language code, which after getting translated into assembly language will have instructions such that some instructions scramble previous instructions (those should be fetched before getting scrambled...here we might have to assume size of instruction pipe, say X).

Is it possible to write such code in a higher level language, if yes can I have an example.

share|improve this question
    
Wait, scramble previous instructions? After they've been processed? And doesn't modern chip architecture/bios disable this type of behaviour at that level, to eliminate self-hiding viruses? –  Clockwork-Muse Feb 1 '12 at 18:10
    
No. I think code is allowed to be modified. Processor will never know that some instruction is modified after it has been fetched. –  hrishikeshp19 Feb 1 '12 at 18:31
2  
Why would you scramble previously executed instructions? What use case would this fulfil? A reverse engineer will look at the code before it runs and analyse that... –  Rory Alsop Feb 1 '12 at 19:20
    
In general, a reverse engineer debugs into the code. Use case might be: instruction that compares activation string to some string, when this instruction is fetched, the instruction ahead of it will scramble the comparison instruction. So, activation string comparison will return yes in normal processing but return no when OllyDbg debugger is used. –  hrishikeshp19 Feb 1 '12 at 21:27
add comment

2 Answers

up vote 3 down vote accepted

In modern computing regardless of a high level language or not the result is the same because it all compiles down to a set of known instructions that the processor can execute.

Since the current generation of processors can’t process encrypted instructions options are limited, and even if this was possible there are several layers in-between that need to be factored in such as operating systems, driver etc.. which perform processor scheduling and memory management to some degree and need to understand what the request is.

Reverse engineering, could be used for many purposes including:

  • Malicious code injection
  • Digital rights, copy protection
  • Extraction of source code, intellectual property

Fortunately we can make reverse engineering harder by:

  • Code signing
  • Obfuscation of strings, resources, entry points and memory

But when you get down to the point of code running on a processor it doesn’t lie.

What you are talking about is in the realm of quantum computing which may offer a solution to your problem. I can’t help you with source code on that one though.

share|improve this answer
add comment

Here's an example in x86 assembly:

MOV AX, 4Ch
MOV [someaddr], AX
MOV AH, 30h
INT 21
CMP AL, 5

This will take advantage of the instruction pipelining of the processor and the failure of debuggers (as they single step through the code) and can as well be an example of an anti-debug measure. This is so because int 21(DOS interrupt) along with an 30h in AH will return the dos version of the platform. However the first two lines have been so crafted that [someaddr] will become 4Ch (someaddr has been so chosen such that it contains the address where 30h has been stored). Thus, 4Ch in AH followed by int 21 will terminate the program (interrupt thingy!). Without the influence of a debugger this won't be a problem as the third line will be prefetched and the first two lines will have no effect. But debuggers will be broken as they single step the code and AH will now contain 4Ch which will of course terminate the program. You asked for the higher level version of it. For that you will want to know how gcc (or any other compiler toolchain) converts C to asm code and then decompile the above code. http://www.programmingforums.org/thread4484.html

But some really cool guys came up with a solution and actually wrote a countermeasure (debugging a loop which has been software-pipelined): read Non-transparent debugging for software-pipelined loops by Hugo Venturini, Frederic Riss, Jean-Claude Fernandez and Miguel Santana.

Another solution to it would be to make the debugger aware of the pipelining.(drop a mail to Oleh!)

share|improve this answer
    
Welcome. Generally, we strive to provide answers that are self-supporting with links for further references. In time, those links might become invalid. Without following links, your answer is of low value. If you can write the relevant information into your answers, that would be very helpful. –  Jeff Ferland May 25 '12 at 14:40
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.