The server of a customer has been hacked last night. It is a standard debian distribution with mysql, apache and ssh for remote access.
The way it has been hacked is very peculiar. The standard URL showed a deface page with link to nethack.alt.org. I have searched entire drive and could not find the files that would belong to this defaced page (bg.jpg ie.). Also, the ssh service and mysql were not accessible. Simple reboot did not help.
I see two possible scenarios:
- Hackers somehow installed a parallel system that got booted instead of the original debian.
- They somehow redirected all traffic to some other server (ip adress too, not just dns)
Does someone have experience with this kind of attack? Where can I look for compromised files?
Note: I will reinstall the entire system from scratch, but I would like to know how they came in, so I can prevent it in the future.
Edit: Additional info. This line is the last in my syslog when the rogue system was booting instead of the real one.
Jan 21 08:33:09 Debian-76-wheezy-64-minimal kernel: [ 17.223513] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
On a normal start the mysql is started at that point:
Jan 21 09:12:12 Debian-76-wheezy-64-minimal mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql