Take the 2-minute tour ×
Information Security Stack Exchange is a question and answer site for Information security professionals. It's 100% free, no registration required.

Our current site is easily DOS'd. I can perform an AB test against it from Apache and it disappears for about 5-10 minutes. We have been having problems with others DOSing the site - to the point where we had to shut it down for 8 hours (recommended by the server host).

The server is hosted by Rackspace. They recommended shutting it down for 8 hours, which when we brought it back up the attacked resumed.

We setup cloud flare for the DNS, however after an AB test with 58,000 requests sent the site was offline again (not as long as originally).

Rackspace is not offering anymore suggestions, and I need to figure out something to do for the time being, switching server hosts is not an option at this point and time (lead developer is leaving in 1 week for a new position).

This specific attack created a lot of connections in the mysql database with the command sleep.

Does anyone have any suggestions or recommendations on ways to protect again DOS attacks?

Would increasing the RAM on the virtual host help at all?

share|improve this question
2  
What type of DoS was this? Memory? Network bandwidth? CPU? The solutions can be quite different for each. –  Rory Alsop Feb 17 '12 at 17:21
    
Not 100% sure - it appeared to be a flood of visitors. Created alot of connections in the mysql database with the command sleep. I am not familiar with DoS attacks so I can't provide tons of details - if you tell me where to look for the details more than happy to find them however. –  Jeff Feb 17 '12 at 17:31
    
popped that into the question to help people notice it –  Rory Alsop Feb 17 '12 at 19:33
1  
Any patterns in those 58000 requests? Similar URLs? Source IPs? It sounds like you'll need your developer to touch the app, but maybe there's a way to make life harder for the guys doing the DoS. BTW when it is "offline" what happens? timeouts? "too many connections?" –  mgjk Feb 17 '12 at 19:43

4 Answers 4

up vote 6 down vote accepted

If the attack is creating a lot of database connections with a sleep command, it sounds like the DoS is not your direct problem, but rather a consequence of the problem. Which is, namely, that your database is sleeping a lot.

I would say there are two possiblities here:

  • First possibility is that this is a built-in function, wherein certain circumstances, your application is explicitly causing the database to sleep (as @Rory mentioned). If this is the case, this is a very bad pattern, and should be removed. There are other solutions, and you should definitely avoid this.
  • More likely, and even worse, is that you have a SQL Injection vulnerability in your website. The good news is that so far, it seems the attackers are only using it to DoS your site. That bad news is that it is probably much worse than that.
    But, more good news, it is a specific bug in your code that should be easily found and fixed by any competent programmer, and not an amorphous and anonymous "DoS" (which could really be any number of things).

Remediation?
First off, figure out which of the above is the case. Should be easy enough, start by asking the programmer if he ever sends a sleep command to the db on purpose. Also should be easy enough to search the codebase for relevant sleep strings.
If it's the first scenario - simply refactor the code to remove the offending command.
Otherwise - and you should probably do this anyway - read up a bit on SQL Injection, and get all the programmers to do likewise. Find and fix the flaws, then do a focused penetration test. (You can get an experienced security pro to help here). Then consider the root cause of such critical security bugs, and start enhancing the overall level of security, via training, code reviews, SDL, pentests, etc. (A security pro can help with this too).

share|improve this answer
    
I asked the developer previously if he had any calls that ran the sleep command, he said no. He actually left the company as of last week - he said that the site is fully protected against SQL injections however. My current plan is to go over it and see what I can find, however it is written in CakePHP which I am not familiar with - have never used php with a framework, or the mvc architecture - so it may take me some time. Thanks. –  Jeff Feb 28 '12 at 16:47
2  
@Jeff I would take any programmer proclamation "the site is fully protected against XXX" with a grain of salt, unless he explained (a) what protections he has in place and (b) how he knows its protected. Especially PHP, and especially with PHP frameworks, it's highly likely there is SQL Injection flaws. –  AviD Feb 29 '12 at 5:47
    
@AviD sql injection depends on the code style the programmer is using. If he's concerned, he'll always do additional "just in case" actions over the data he receives. –  tftd Mar 2 '12 at 13:59
    
@tftd I dont understand your comment? Both the first part ("code style"?) and the last part (additional actions over the data) dont seem to be relevant, or at least I dont understand your meaning. –  AviD Mar 2 '12 at 14:03
1  
@tftd He didn't say he didn't know what SQLi was; he said he didn't understand what you were trying to say. I don't either, to be entirely honest. –  Parthian Shot Jul 12 at 4:20

There are a couple of things you can do to prevent DOS/DDOS.
First I would recommend using an autoban firewall. I've been using Fail2Ban for some time now. Most of my problems with DDOS/DOS attacks to SSH,FTP,BIND and etc were solved. What fail2ban actually does is it's scanning a log file and when a regex pattern matches X times it bans the person. With a little more work you can make it read almost any type of log files.

Second there were a couple of mods for apache that prevent DDOS/DOS attacks. One of them was mod_evasive. I used it for a while but fail2ban worked for me best (for now) so I didn't quite get into mod_evasive.

Another possibility is to use Nginx as a front-end which would proxy-forward connection requests to your Apache server. Then set Nginx's HttpLimitZoneModule (info) to a desired number. This will reduce the maximum incoming connection from a single IP but it won't prevent DDOS.

Finally - close all of your unused ports. Leave only ports that need to be public as port 21,53,80,443 etc. This will limit the possibilities where the attacker may target.

Edit:
Make sure you don't get ping flood by setting some iptables rules that limit icmp. Here's a link which explains how ping flood works. Try looking at the pictures first then the text will make more sense to you.
If you're using iptables this command may help you prevent ping/icmp flood:

iptables -A INPUT -p icmp -m limit --limit 6/s --limit-burst 1 -j ACCEPT
iptables -A INPUT -p icmp -j DROP

The first one will limit icmp requests up to 6 requests per second. Afterwords the person will not receive any more responses. This will eliminate bots that are doing a cycle:

for(i=0; i<1000000; i++){ 
   some evil code that will bring your server down 
   either by pinging or just connecting to random ports. 
};
share|improve this answer
1  
+1 Clear, articulate answer with nice links. Also, yeah Fail2Ban is a good protection for DoS. –  Shivam Feb 17 '12 at 23:50

This article is a good start for the layman. It ultimately suggests that you hire a security expert.

If that's not an option (if you are expected to be the expert), then I have read that an effective DOS mitigation strategy involves:

  • Having cutting-edge IDS tools (3rd party or even homegrown) to detect the onset of a DOS attack (a sudden spike in traffic, etc.); and
  • Devoting someone full-time (at least, for a while) to monitoring network traffic and blocking IP addresses that are connected to the server redundantly (more than once) --> see this article for more info

Without knowing who you are/work for, it is hard to guess whether you are being consistently re-attacked by the same parties. If you're a big firm, its probably multiple (disparate) groups trying to shut you down. If you're smaller, it's probably the same entity/group attacking you. If that's the case, then if you commit to aggressively blocking violating IPs in this fashion for several weeks, chances are you will grind the attackers down over time and thwart them.

The big thing in security is to make the cost of acquiring the asset greater than the value of the asset itself.

And, if you're a big company, you can afford to hire a true expert/consultant who knows leagues more than I do ;-)

share|improve this answer
    
Thanks, we are a small company and it is more than likely the same party over and over again. Unfortunately I don't make the final decision whenever it comes to costs / hiring - but it is a decent size retail site which generates large revenue for the company so the cost to prevent should out way the no-income followed by down time. Thanks for your reply. –  Jeff Feb 17 '12 at 19:39

Have a read of this question for some more information on how DoS attacks work.

For this specific case, the attack is using a function you are allowing (sleep) to force wait cycles until your database can no longer respond.

A solution for you may be to disallow the sleep command entirely, unless you have a need for it. A better solution is to prevent all connection to your database except through stored procedures, so that only those commands you need to run your application are allowed. This will make it that much harder for an attacker to find and exploit a hole.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.