How to analyze a rootkit?

Question

The other a friend of mine sent me a rootkit/code he found on a server that was exploited (php cgi exploit).

Now I was thinking of having a look at it, but I was wondering if there are some basic steps to follow to analyze these things?

I'm not talking about the obvious stuff like using a sandbox that's not attached to the network. But more how one does a thorough step by step analysis and how this normally is documented.

Answer 1

As part of a CTF challenge that I run, we had some reverse-engineering challenges last year. I posted some advice on how to perform the analysis (verified by folks that do this on a daily basis). The blogpost is here. The links point to tutorials for IDA and OllyDbg, two of the most popular tools for such analysis and there's a nice paper from one of the AV vendors. From memory the tutorials were step by step.

There's also some rootkit analysis videos on http://www.securitytube.net/.

Answer 2

Firstly, a php/cgi based programm might not be an rootkit - rootkits are usually assocated with malware that hides itself in a kernel, not just a malwares.

If you sure are that it is a rootkit, you have two choices:

• Offline analysis - given a image of the powered down system, you can analyse what files it has changed. Similarly, you can analyze the binaries the same way you analyze other malwares.
• Online analysis - this is what rootkits try to prevent on themselfs, or on the malwares they hide; however, often you can perform some analysis.

Unlike normal malwares, binary instrumentation or debugging of rootkits is much harder as they are running in the same security level as the rest of the kernel and you have to attach a debugger to the kernel. You might choose to use KGDB for example.

Offline analysis is much like reverse engineering other malwares, but with a mind to the fact that the codes is relative to the kernel, etc. Things may be many more complex. For this, IDA has already been mention. This is a standard industry tool for such things.

So the process is not so different to a normal malwares, really.