I'm a moderator of a major bulletin board. When a baddie shows up, we block their IP address; it works, at least until they find a new one. Why can't a protocol be developed for the world's routers to combat DDoS, whether by IP addresses or message content or something else, to stop DDoS in its tracks? There's clearly an answer, or else it would have already been done. Can someone give an executive summary of why it can't be done? E.g., it would require some central authority that doesn't exist.
Let's say that you run a shop. Every day, you might get a few hundred customers.
One day, you get tens of thousands of people coming in, who get in the check-out line, buys a trinket, and then gets right back in line to repeat.
Obviously, you are losing business from authentic customers who must wait hours in line.
Now, you hire a security guard at the entrance to verify that these customers satisfy some criteria.
However, there's still tens of thousands of people who want to get in. The only difference is that now, everyone must go through security.
You notice that, from the authentic customer's perspective, you still wait hours in line, just that now it's just to get through security!
TL; DR The multiple source IPs are what makes them so hard to defend against.
For the longer answer we look at the name. DDoS attack. That first D stands for distributed. In other words, there is no one IP to block. Or two, or three, or even four.. There are hundreds or thousands of unique IPs.
Usually DDoS attacks originate from a hacker in control of a botnet or network of zombie machines. The attacker will issue a command to all the bots instructing them to make requests for a particular resource / URI. The large number of requests overwhelms the server and takes it down.
In addition to @Hollowproc's excellent answer, the actual "addresses" being used as sources are often spoofed in an attack like this. An attacking host can pretend to be any number of other IPs, especially in a UDP-based attack such as is used against DNS providers.
There's a solution for this called BCP 38, or Network Ingress Filtering. If all the world got together, had a coke, and implemented controls to block spoofed addresses where they enter the network, these attacks couldn't spoof traffic any more. The Dyn folks themselves mention it as a helpful defense.
Note that implementing a defense at many source points has the advantage of being scalable; the individual block requirements are not onerous in size. However, they are onerous in that more people must do the right thing about something that does not directly and negatively impact them... human nature has, for over a decade, prevented this solution from reaching critical mass.
It is possible that the growing impact of DDoS attacks will spur greater adoption of Network Ingress Filtering... The Internet treats damage as censorship, and routes around it :)
The problem with a DDoS is two part:
1) Since the bots are so many, they don't have to have a request throttle as high as a single bot, and are therefore not as easy to recognize as bots.
2) All you see is IP addresses (and
User-Agents depending on how you filter bot traffic). Any IP address could be a DDoS bot and any IP address could be a legitimate visitor. Some IP addresses will have both a DDoS bot and a legitimate visitor. What do you do?
Let's say your site can handle 1000 req/s and a visitor never makes more than 10 req/s. One bot at 100 req/s is easy to block, ten bots at 100 req/s are easy to block. But 200 bots at 5 req/s are difficult to block, because they are behaving properly.
200 bots at 100 req/s are difficult to block too, because they can't even make more than 5 req/s, making it appear that they behave properly. This situation is far worse than 200 bots at really 5 req/s, because a visitor is now 10 among 10010 requests trying to squeeze into the connection rather than the more manageable 10 among 1010 that are successfully reaching the server.
1000 bots at 100 req/s would make it improbable for each real visitor to connect to the site at all. And an attack of this magnitude is going to cause trouble along elsewhere as well.
A strong defense for your site is a strong weapon for an attacker:
If your site blocks IP addresses (or even specific browsers on the IPs) if they misbehave, someone might decide to abuse your defense to cause a DoS attack on the client side.
Example: Mallory creates a webpage that has a hundred "images" with
margin-left: -1000em; width: 1px; height: 1px; and all of those images are some "sensitive" URLs on your site. When a visitor visits Mallory's page, they will send 100 abusive requests to your server and will therefore become blocked.
So getting a more aggressive defense against (D)DoS attacks will also cause another vulnerability.
A "smart" defense such as CAPTCHAs (to give the visitors a chance to prove that they are also visitors and not just malicious bots) will also have the side effect of actually requiring resources of the server.
Uploading images when the connection is clogged up doesn't sound like a very good idea. Neither does remembering a huge number of sessions for the CAPTCHAs, CAPTCHAs that won't be answered, partially because the images couldn't get sent, partially because the majority of visitors are non-human.
And on a (highly or uncached) dynamic site, you'd be fighting fire with gasoline.
Suppose you are hosting a service of some kind, whose main purpose is to serve a particular geographic location, and let's assume you did this by some access rules based on IP addresses.
Now if UDP is taken in that context, anyone with a knowledge of a packet crafting tool (e.g. scapy) can spoof the legitimate IP that are allowed access, and if you opt for the blocking approach, you will be blocking the legitimate users from the accessing the service.
TCP Similarly spoofed TCP (e.g. syn flood) DDoS attacks can make a genuine user suffer if the active blacklisting approach is followed.
So to deal with DDoS it's better to apply careful filtering, using devices intelligent enough to distinguish between genuine and attack traffic by DPI or some other algorithms.
CDN, NAT, PROXY
If users are behind a CDN or something like that, blocking one user may make the whole bunch behind the proxy suffer.
Moreover, "whether by IP addresses or message content or something else," as you mentioned, for routers to be capable of filtering on a content basis, that would hinder its routing performance. But yes, there are still in routers to do that (e.g. NBAR), and all which one can apply inside his premises.
And blocking should be on a more specific basis.
Others have made great answers regarding the technical challenges around mitigating DDoS, my answer here though is going to focus on what happens once you've built the capability to filter the DDoS by blocking large number of IP addresses onto your site.
Blocking large number of IP addresses is not always desirable. By blocking large number of IP addresses due to a large DDoS, you could be blocking a large number of legitimate users who might share those IP address as an undesirable side effect (e.g. Tor, proxy users, universities, shared household, ISPs that use NAT to save public IP addresses).
This might not be so much of a problem for small personal sites, but for a number of sites where the users have legitimate need for serious anonymity, say sites catering to political activists, LGBT, female services in oppressive countries, domestic abuse, etc. Simple IP blocking is essentially falling into the attacker's ploy for these sites. By denying service to anonymity users from accessing your service, you may be preventing your most vulnerable legitimate users from safely accessing your site, and that achieves the attacker's goal just as well as the original DDoS.
There isn't a simple or single solution to DDOS attacks, because they can be performed in lots of different ways. The nature the technology behind the Internet doesn't lend itself to anything but being wide open. There are a lot of patches and work arounds to try and shore up security and mitigate a lot of problems. In the balance, it's a lot more difficult to protect a site or network against an attack than it is to attack. That's just the state of security today.
For network level attack (like again DNS most recently for Dyn), Network Ingress Filtering as mentioned before, would've been helpful. That at least helps with the spoofing problem.
A more pressing issue with this type of attack, though, is the scale. With the number of infected systems in a botnet for a sizable attack reaching tens of thousands, if not hundreds of thousands of systems, doing IP-based blocking is not reasonable. Have far up the network chain will you need to block to survive if you do blocking? The Krebs DDOS attack was claimed to be in the 600Gbs range. Can you or your provider handle that (or even just a more typical in my experience 10-120Gbs) Regardless, you are just playing whack-a-mole at that point, since once blocked your attacker will likely switch to another set of exploited machines with different IPs.
If you do decide to play the IP reputation game <cough>CloudFlare<cough>, you can do stupid things like end up blocking large providers- e.g. Pokemon Go dropping most/all traffic from Belgium. Again, this is just whack-a-mole and not a viable solution.
Let's talk higher up the stack. Web service attacks, like credential stuffing or scraping, will frequently look like legitimate traffic. Junior-league script kiddies will have out of whack browser signature you could look for, but things like Sentry MBA or even PhantomJS will mimic proper browsers that will defeat this simplistic HTTP header/browser ID finger printing. Better attackers will even simulate proper mouse and keyboard use, further obscuring their automated nature.
Captchas are expensive both from an implementation standpoint (either resources on your servers, or outsourcing $$). Simple ones can be defeated with reasonable image detection algorithms. More complex captchas start pissing off users and may cause accessibility issues for visually impaired users. There are also systems that effectively defeat captchas through mechanical turk either directly (hoards of people paid pennies on the captcha) or indirectly (captcha from your site being answered by an unsuspecting user on another site).
Anyway, as attacks are multi-pronged, you need a multi-pronged approach to defense. Large operators have even gone on the offensive, as Microsoft has worked with the FBI to take over and shutdown botnets. Unfortunately, IoT has just opened up a whole new set of systems that come out of the box open for exploitation.
The genius of DDoS attacks stems from the fact that the traffic comes from potentially LEGITIMATE IPs of real customers.
Let's say a normal person living in the US could have his router compromised and have it used for DDoS. The victim company blocked the IP address completely, now that person cannot connect to the company's web service at all because the company blocked the IP entirely, cutting off a potentially valid IP from ever accessing their servers again.
This was a simple example but imagine a university having a few NAT IPs for web browsing and a company blocked some of those NAT IPs during a DDoS mitigation. Now tens of thousands of people on that campus could lose connectivity to that company's servers which is catastrophic for business.
Not to mention large enough DDoS attacks can employ a crazy amount of different IP addresses.
It is not easy to distinguish between a normal traffic and a DDoS traffic.
DDoS can be explained in easy words as -
As a human being, i can only discuss (have a chit chat) things with one person at a time and if 10's of people speaks to me at a same time, then i will not be able to answer any of them and i will be in not available state for all of them.
Attackers usually generate the DDoS attack from compromised machines (also known as **bot). It may be case that at the time of writing the answer of your question, my machine may involve in ddos attack to some destination (if my machine is compromised, although chance of its is very less).
As explained, there is no black and white solution available for controlling (or blocking) ddos attack.
As explained by @gowenfawr, if ddos attack pattern is of udp, then implementation of URF at the ISP level across the internet, can help in blocking the spoofed traffic and hence help in controlling the ddos attack.
True story here
We used to sell a small camera system. Was by a big name brand and the camera wasn't spectacular, but still cost a decent amount. One day, we had someone call us up about a charge on their credit card. Turned out someone had gotten it and used to buy one of these cameras and had it shipped to someone who probably aggregates them and either fences them directly or bulk ships them to the frausters.
Being an enterprising admin, I dug into the logs to figure out where the fraud was coming from. I found one from an African IP, but the rest of the suspicious orders all had American IPs. In fact, they were all being proxied through compromised servers in the same datacenter our web servers were in. Beyond reporting it to the host, there wasn't anything to be done. Whomever did this was pulling the strings via compromised machines. There's just no way to guard against this sort of thing by looking at the network source itself. You never know where or when someone will get compromised and their IP turned into a weapon.
The recent attack on Dyn showed just how stupid simple that is now
The botnet, made up of devices like home Wi-Fi routers and Internet protocol video cameras, is sending massive numbers of requests to Dyn's DNS service. Those requests look legitimate, so it's difficult for Dyn's systems to screen them out from normal domain name lookup requests.
They're tough attacks to stop because they often get channeled through recursive providers. They're not cacheable because of the random prefix. We started seeing random prefix attacks like these three years ago, and they remain a very common attack. If IoT devices are being used, that would explain the size and scale [and how the attack] would affect: someone the size of Dyn.
With IPv6 taking off, soon you could have billions of devices, each with its own IP, to work with. The scope is too large to effectively filter against.
A protocol that allows "someone" telling internet infrastructure/backbone parts to stop accepting/forwarding traffic from certain addresses just means that this "someone" would not even need to employ DDOS techniques to put anyone they want off line. Without a "central authority that doesn't exist", it would be hard to agree on who should be trusted with that kind of power.