What I mean is what protocol are best to DOS something like TCP, UDP, HTTP?
You are probably referring to DoS attacks that are carried out by excessively sending out i.e. flooding some type of packets, consuming all resources of the target. From that perspective the less resources you need to use compared to the resources exhausted on the other end makes the attack more powerful. In reality, this type of DoS is the least powerful.
In the most powerful DoS attacks there must be some vulnerability that can be used for crashing or shutting down an entire software/system with a single carefully crafted packet or request. It's easy to find many such vulnerabilities by listing DoS vulnerabilities and ordering them by severity (score).
Let's take an excellent historical example:
ping -c 1 -p 2B2B2B415448290D 198.51.100.99
- The pattern
2B2B2B41544829is the hex code equivalent of
+++ATH0ASCII string and
0Dis a carriage return character. The count
-c 1is here to emphasize that a single packet can be efficient.
- The target answers to this ICMP echo request with an echo reply having the same contents.
- Some modems treated the
+++as an escape string to enter the command mode (for commands intended for the modem itself), even without the time guard interval (Time Independent Escape Sequence, TIES). This was to avoid licensing Hayes's patent.
- These are Hayes command set a.k.a.
ATcommands, and the
H0hook status "hangs up the phone, ending any call in progress". The target immediately loses Internet connection.
Of course this example is not so useful today, because modems are becoming rare and the problem has been mitigated long before e.g. using different escape sequences. Critical DoS vulnerabilities like those found on the vulnerability database are also fixed on a high priority. Despite that, this is the most powerful type of DoS, as it takes the least amount of own resources having a huge/lethal impact on the target.
The most efficient DOS attack is one that requests the server to use a lot of resources, while not requiring a lot from the client
eg: ask for the middle row from a list of a billion which the server must sort first. ask for a slow computation. force the server to retrieve a very large DNS record. exploit a badly written regex - or supply your own. exploit a server bug that forces the server process to restart.