4

ATTACK VECTOR
Monthly hits via a globally DDoS attack. Resolved with Cloudflare.

Attacker can now somehow unmount network drive links and reset MySQL user permissions for applications.

Attack is resolved by simply by issuing these commands:

GRANT ALL PRIVILEGES ON application.* TO application_user;
FLUSH PRIVILEGES;

I also need to remount the remote drive using sshfs

SYSTEM DESCRIPTION
My system operates with 2 servers.

A server has MySQL, Memcache, Nginx, and a lot of static content.
B server has Apache, PHP5-FPM, and mounts the static drive from A to serve some shared content. This is for imaging purposes.

A server and B server only talk on ports 80 and 443 to Cloudflare because we have been DDoS'd before. As far as I know, their true IPs are unknown.

A server does NOT allow foreign connections on 3306 for ANY user.
All MySQL passwords are 32 characters long and include non-alphanumeric characters ($@_'")
The passwords of users DO NOT change and the attacker has NEVER been able to provide me evidence they can read the database.

A and B server do not accept SSH on port 22 and their passwords are 64 non-alphanumeric characters. I have never seen logs that show an unfamiliar connection was made.

Application-level users do not have the ability to access mysql.user and they cannot make changes to this table. An SQL injection is unlikely because the attacker does not drop tables.

  • Don't know about any method to reset permissions other than intrusion. They could have entered your systems through unpatched vulnerabilities and/or insecure configuration. Dynamic web content like your PHP might be vulnerable. Try to dump the memory, check running processes and make an image of your systems and analyze logs, code etc. offline if possible and try to find the points of intrusion. You could also log MySQL queries to look for suspicious actions. Also, avoid SSH with passwords and use public key auth if possible. With passwords, they could MITM your SSH. – Archimedix Mar 4 '15 at 6:34
  • Oh, and you should have backups, of course, before someone kills your data. – Archimedix Mar 4 '15 at 6:36
1

A reboot of server A.

What's the uptime of server A ? Is it possible that it is having an disk error / getting restarted? In that case, the sshfs mount would obviously require a remount. And sometimes there are boot scripts in place that could be resetting some credentials to the default ones…

PS: You don't need the FLUSH PRIVILEGES when using the GRANT statement. Only when directly changing the mysql.user table (INSERT, UPDATE…) it is needed.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.