8

I originally posted a question here since I was not familiar with an actual distributed denial of service attack (DDoS) on my server. I noticed that my server is very slow in responding to HTTP requests whereas HTTPS requests are fairly good (not as good as expected though).

By doing a bit of analysis and looking at the dmesg and the growing size of apache's error.log (over 5GB), I found out that hey I'm receiving a DDoS attack. I shutdown Apache and did a netstat, I found out that all the SYN flood is gone.

In summary:

The problem was that remote servers were initiation requests to Apache on port 80 and would leave the tcp connection open and thus exhausting Apache's resources.

What I did:

I checked if I had any Apache updates, but I was up to date.

To make sure that my server did not run a bot to communicate with a command and control server, I replaced the current volume image with a very old one at the time I did not have any problems. As soon as I started the server, I received the attacks as before, so this wasn't an issue.

I installed CSF, although I was hesitated because I could not initially trust this software. After starting CSF's SYN flood prevention, I noticed some improvement on HTTP requests but not much.

What I finally did was to change the IP of the server and the attack was gone. By the way, iptables rules were not that effective, because I ended up blocking legitimate traffic.

What's still a question for me:

  • Why HTTPS requests were being handled in a timely manner?
  • Why servers are still easily vulnerable to such attacks despite tons of research in this area?
  • How can I prevent this problem in future?
Improve this question
4
  • 1
    Your second question is a good one... stay tuned.
    – Polynomial
    Commented Oct 22, 2012 at 15:26
  • 1
    In the meantime, take a look at CloudFlare, which has a free service that can mitigate most small DDoS attacks.
    – Polynomial
    Commented Oct 22, 2012 at 15:53
  • @Polynomial in general I don't like the idea of sending my traffic to a third party for analysis and mitigation, though it's a nice feature.
    – hsnm
    Commented Oct 22, 2012 at 17:56
  • 1
    for application layer ddos i would look up Apache module : Mod_evasive it is open-source and you could dig out ways of preventing or apply the module to your server
    – user24303
    Commented Apr 4, 2013 at 16:55

1 Answer 1

Reset to default
6

  1. Requests to port 443 were likely faster because you were being DDoSed on port 80. Apache preforks processes. To service port 80, Apache needed to spawn a new process. To service 443, Apache was using an existing process. (I think... must check later)

  2. DDoS mitigation is still a hard problem. You need to discern legit traffic from fake traffic, and you need to do it with as little interaction as possible.

  3. If this is a for-profit service, a professional web server admin and/or a DDoS mitigation service would be the way to go.

If you are a web server admin, then digging into the features of the tools e.g., CSF and Apache mod_security can help. Attackers are focused on trying to look like legit traffic. I'm an admin myself and this kind of attack is a serious PITA to protect against. Getting a handle on the signature of the traffic and filtering appropriately can take time. Fortunately, today, the DDoS tools don't respond quickly, so they'll move on to the next target or just give up for a while.

Note that there is a lot of sorta-legit traffic which looks like DoS attacks. Badly behaved spiders are the most annoying to me. People linking graphics into social media can look like a DDoS, with dozens or hundreds of hits per second (from unique IPs) if somebody deep links a graphic on a popular forum.

Be sure when you block somebody with IPTables, don't "reject" their traffic. Drop it. It slows things down.

I look forward to reading other responses on this topic.

Improve this answer
2
  • Yes I see the difficulty in DDoS problem when the traffic cannot be easily distinguished. Probably pattern recognition or anomaly detection tools will have a hard time. I think the best way to go is to design the system such that the attacks don't at least completely interrupt the service. Maybe having multiple servers with a load balancer can help.
    – hsnm
    Commented Oct 22, 2012 at 17:58
  • @hsnm What you've described here is very similar to the reverse proxy suggested earlier by Polynomial. CF is a good free service, as far as free services go, but I agree with mgjk - for a commercial site I would look into a commercial grade solution. Also, compared to multiple server setup, reverse proxy can provide more scalability. Even multiple servers can still be over-flooded (easily so) but a global network can "grow" instantly as needed. Finally, specifically for SynFlood, I can suggest using SynCookies to minimize resource depletion. (We do it and it helps with smaller attacks)
    – Igal Zeifman
    Commented Oct 23, 2012 at 14:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .