For this I am assuming we have a server that is of a client or a friend, so we the helper are not innately familiar with exactly how things should be be.

What are good commands and ways to check to see if anything bad has been going on?

I know we have: the 'history' command, to see commands that have been entered and the 'last' command to see who else has been logging in.

Is there a general guide or set of best practices I can be referred to? I am using Ubuntu myself.

  • At least you should check that package fingerprints are more or less intact (hopefully they have debsums installed), see debian-handbook.info/browse/stable/… . Also it would be a good policy to list all files not owned by any package and then see what those files actually are, especially in case they're executable binaries.
    – Sami Laine
    Dec 30 '13 at 20:53
  • I agree with symcbean but we already have serverfault.com/questions/171893/… for your perusal.
    – user400
    Dec 31 '13 at 10:24

If you have reason to believe that the system may be compromised then wipe the server and restore from last known good backup.

By all means take an image to see if you can determine how the system was compromised and plug the vulnerability. But this is not an easy process, particularly if you have no experience at this kind of thing - indeed if you attempt this yourself then it's likely that such intervention may occlude the nature of the compromise - certainly the "evidence" would be considered severely tainted by any law enforcement / jurisdiction.

Validating the package checksums is a starting point — but not nearly as effective as a properly managed file integrity checking regime (tripwire, LIDS etc). Logs are good place to look too. But you have to understand what these tools are telling you.

There's lots of things you should be doing before a system is compromised: configuring the permissions, firewall, optionally installing additional security measures such as mod_security, suhosin, fail2ban etc. Part of the maintenance of a server should be running a rootkit detector on it regularly. For a system/cluster of significant value you should also be carrying out periodic vulnerability scanning. But there's no point in doing any of this after the event.

  • +1 for mentioning LIDS, omg, blast from the past. Seriously though, don't use LIDS. Pretty poorly documented and not widely used. Last message on the site is from 2010. RPM provides md5sums for all files, and there is also debsum: manpages.ubuntu.com/manpages/natty/man1/debsums.1.html.
    – dmourati
    Jan 1 '14 at 3:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy