5

Let's say I hire someone (freelance), for IT work on an Ubuntu server. The server has a good firewall, I run clamav, chkrootkit & rkhunter scans weekly, and I try to keep good practices.

The thing is, if I suspect that he installed some backdoor on the machine, how would you search for it? He had root access and plenty of time, so...

I know this theme has been discussed earlier, but most were theoretical rather than practical answers.

NOTE: This is on a remote server, so I'm not able to do hardware-based analysis.

  • 4
    Really, you can't. If you suspect a server's been compromised, a clean install is the best option. There are just too many ways to hide a backdoor for there to be any reliable way of searching for them. You can't even necessarily trust the output of any commands you run. – tlng05 Jun 8 '16 at 1:05
  • @tlng05 I'm able to run a rescue image, which I know is legitimate, and then run diagnostics. – Agustín Covarrubias Jun 8 '16 at 1:08
  • 3
    That helps, but the general idea still applies. There are just too many ways a backdoor can hide and many are simplistic enough to not be picked up by scanners. You need to be able to trust the people you give root access to. – tlng05 Jun 8 '16 at 1:14
  • @tlng05 should I delete this question? hahaha – Agustín Covarrubias Jun 8 '16 at 1:15
  • 3
    Don't delete the question, it's useful and interesting to other people. – Mike Ounsworth Jun 8 '16 at 2:36
8

To expand on @tlng05's point, rootkit authors are professionals, often with decades of experience. Unless you have decades of experience fighting rootkits, then they'll know about more hiding places than you do :P

If you suspect there's a rootkit, then your only choice is to wipe the system. This is a classic example of Ken Thompson's reflection on trusting trust "you have to trust the people, because you'll never find the exploit".

Next time, I suggest being more strict about who you give root access to.

  • 1
    While this is true if the defender hasn't done any preparation, you can certainly detect a rootkit if you use AIDE or a similar offline integrity checker. This is of course assuming a software rootkit. – forest Dec 8 '18 at 4:00
2

That highly depends on the complexity of the attack. If it's a simple backdoor, you have a chance of finding/eliminating the backdoor with the following method:

1.

netstat -antp

search for the sending port/program and remember the pid + ip.

2.

lsof -p <the pid>

look up the dies which are showing up. (These are the files used by the sending process)

3.

  • Try deleting these files
  • Block the IP via. iptables (tutorial)
  • Change passwords of all your users
  • Pray

Notice, this is NOT a guaranteed way to remove the backdoor. Like tlng05 already mentioned, the most secure method would be to backup your important files and wipe the complete system. (I strongly recommend doing so too)

1

You can dump the memory on your running server and do the analysis using volatility. If this server has not rebooted then you might find something interesting such as command history, strange processes, network connection ... using existing modules. I can not go into details because this is a long story and training skills to do this.

Actually, people in forensic teams use similar tools like this to investigate incidents esp IR teams.

protected by Community Jul 26 at 14:10

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.